Mahiruoshi

Posted on Edited on
Symbols count in article: 118 Reading time ≈ 1 mins.

♬月火水木金土日♬
♬毎日が Workday♬
Hi, 我是Mahiroshi, 正如网名, 是个偶像宅
I am currently a third-year undergraduate student at Nanyang Technological University, majoring in Computer Science.
I am currently interested in the topic of CADS in medical image processing at the MAVIT LAB. Before that, I join the DPNM lab during Postech Summer Program and S-Lab for Advanced Intelligence during the second year in NTU.
I am searching collaboration about AI implementation in Medical Image area. Please condact me if you have interest in it.

关注Mahiroshi喵！关注爱马仕谢谢喵！
cv
Summary of my Github Project

Posted on In
Symbols count in article: 9.1k Reading time ≈ 33 mins.

awesome-domain-adaptation

MIT License

This repo is a collection of AWESOME things about domain adaptation, including papers, code, etc. Feel free to star and fork.

Contents

Papers

Survey

Arxiv

  • Video Unsupervised Domain Adaptation with Deep Learning: A Comprehensive Survey [17 Nov 2022] [project]
  • A Survey on Deep Domain Adaptation for LiDAR Perception [7 Jun 2021]
  • A Comprehensive Survey on Transfer Learning [7 Nov 2019]
  • Transfer Adaptation Learning: A Decade Survey [12 Mar 2019]
  • A review of single-source unsupervised domain adaptation [16 Jan 2019]
  • An introduction to domain adaptation and transfer learning [31 Dec 2018]
  • A Survey of Unsupervised Deep Domain Adaptation [6 Dec 2018]
  • Transfer Learning for Cross-Dataset Recognition: A Survey [2017]
  • Domain Adaptation for Visual Applications: A Comprehensive Survey [2017]

Journal

  • Survey on Unsupervised Domain Adaptation for Semantic Segmentation for Visual Perception in Automated Driving [IEEE Access 2023]
  • A Review of Single-Source Deep Unsupervised Visual Domain Adaptation [TNNLS 2020]
  • Deep Visual Domain Adaptation: A Survey [Neurocomputing 2018]
  • A Survey on Deep Transfer Learning [ICANN2018]
  • Visual domain adaptation: A survey of recent advances [2015]

Theory

Arxiv

Conference

  • Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift [NeurIPS 2020]
  • Bridging Theory and Algorithm for Domain Adaptation [ICML2019] [Pytorch]
  • On Learning Invariant Representation for Domain Adaptation [ICML2019] [code]
  • Unsupervised Domain Adaptation Based on Source-guided Discrepancy [AAAI2019]
  • Learning Bounds for Domain Adaptation [NIPS2007]
  • Analysis of Representations for Domain Adaptation [NIPS2006]

Journal

  • On a Regularization of Unsupervised Domain Adaptation in RKHS [ACHA2021]
  • Unsupervised Multi-Class Domain Adaptation: Theory, Algorithms, and Practice [TPAMI2020] [PyTroch]
  • On generalization in moment-based domain adaptation [AMAI2020]
  • A theory of learning from different domains [ML2010]

Explainable

Conference

Unsupervised DA

Adversarial Methods

Conference

Journal

  • Incremental Unsupervised Domain-Adversarial Training of Neural Networks [TNNLS 2020]
  • Unsupervised Multi-Class Domain Adaptation: Theory, Algorithms, and Practice [TPAMI2020] [PyTroch]
  • Adversarial Learning and Interpolation Consistency for Unsupervised Domain Adaptation [IEEE ACCESS]
  • TarGAN: Generating target data with class labels for unsupervised domain adaptation [Knowledge-Based Systems]

Arxiv

Distance-based Methods

Journal

  • Transferable Representation Learning with Deep Adaptation Networks [TPAMI]
  • Robust unsupervised domain adaptation for neural networks via moment alignment [InfSc2019]

Conference

  • Domain Conditioned Adaptation Network [AAAI2020] [Pytorch]
  • HoMM: Higher-order Moment Matching for Unsupervised Domain Adaptation [AAAI2020] [Tensorflow]
  • Normalized Wasserstein for Mixture Distributions With Applications in Adversarial Learning and Domain Adaptation [ICCV2019]
  • Joint Domain Alignment and Discriminative Feature Learning for Unsupervised Deep Domain Adaptation [AAAI2019]
  • Residual Parameter Transfer for Deep Domain Adaptation [CVPR2018]
  • Deep Asymmetric Transfer Network for Unbalanced Domain Adaptation [AAAI2018]
  • Central Moment Discrepancy for Unsupervised Domain Adaptation [ICLR2017], [InfSc2019], [code]
  • Deep CORAL: Correlation Alignment for Deep Domain Adaptation [ECCV2016]
  • Learning Transferable Features with Deep Adaptation Networks [ICML2015][code]
  • Unsupervised Domain Adaptation with Residual Transfer Networks [NIPS2016] [code]
  • Deep Transfer Learning with Joint Adaptation Networks [ICML2017] [code]

Arxiv

  • Deep Domain Confusion: Maximizing for Domain Invariance [Arxiv 2014]

Information-based Methods

  • Hypothesis Disparity Regularized Mutual Information Maximization [AAAI2021]

Optimal Transport

Conference

  • Global-Local Regularization Via Distributional Robustness [AISTATS2023] [Pytorch]
  • MOST: Multi-Source Domain Adaptation via Optimal Transport for Student-Teacher Learning [UAI2021]
  • LAMDA: Label Matching Deep Domain Adaptation [ICML2021]
  • TIDOT: A Teacher Imitation Learning Approach for Domain Adaptation with Optimal Transport [IJCAI2021]
  • Unbalanced minibatch Optimal Transport; applications to Domain Adaptation [ICML2021] [Pytorch]
  • Graph Optimal Transport for Cross-Domain Alignment [ICML2020]
  • Margin-aware Adversarial Domain Adaptation with Optimal Transport [ICML2020] [code]
  • Metric Learning in Optimal Transport for Domain Adaptation [IJCAI2020]
  • Reliable Weighted Optimal Transport for Unsupervised Domain Adaptation [CVPR2020]
  • Enhanced Transport Distance for Unsupervised Domain Adaptation [CVPR2020] [Pytorch]
  • Differentially Private Optimal Transport: Application to Domain Adaptation [IJCAI2019]
  • DeepJDOT: Deep Joint distribution optimal transport for unsupervised domain adaptation [ECCV2018] [Keras]
  • Joint Distribution Optimal Transportation for Domain Adaptation [NIPS2017] [python] [Python Optimal Transport Library]

Arxiv

  • CDOT: Continuous Domain Adaptation using Optimal Transport [20 Sep 2019]

Incremental Methods

  • Incremental Unsupervised Domain-Adversarial Training of Neural Networks [TNNLS 2020]

Semi-Supervised-Learning-Based Methods

  • Label Propagation with Augmented Anchors: A Simple Semi-Supervised Learning baseline for Unsupervised Domain Adaptation [ECCV2020]
  • Semi-supervised Models are Strong Unsupervised Domain Adaptation Learners [arXiv 2021][Pytorch]

Self-training-Based Methods

Arxiv

Self-Supervised Methods

Conference

  • Self-Supervised CycleGAN for Object-Preserving Image-to-Image Domain Adaptation [ECCV2020]

Arxiv

Transformer-based Methods

Conference

Other Methods

Conference

  • Prior Knowledge Guided Unsupervised Domain Adaptation [ECCV2022] [Pytorch]
  • Revisiting Unsupervised Domain Adaptation Models: a Smoothness Perspective [ACCV2022] [Pytorch]
  • Reducing the Covariate Shift by Mirror Samples in Cross Domain Alignment [NeurIPS2021]
  • Pareto Domain Adaptation [NeurIPS2021]
  • ToAlign: Task-Oriented Alignment for Unsupervised Domain Adaptation [NeurIPS2021]
  • A Prototype-Oriented Framework for Unsupervised Domain Adaptation [NeurIPS2021]
  • Understanding the Limits of Unsupervised Domain Adaptation via Data Poisoning [NeurIPS2021]
  • SENTRY: Selective Entropy Optimization via Committee Consistency for Unsupervised Domain Adaptation [ICCV2021]
  • Transporting Causal Mechanisms for Unsupervised Domain Adaptation [ICCV2021]
  • Semantic Concentration for Domain Adaptation [ICCV2021]
  • FixBi: Bridging Domain Spaces for Unsupervised Domain Adaptation [CVPR2021]
  • Domain Adaptation With Auxiliary Target Domain-Oriented Classifier [CVPR2021]
  • Conditional Bures Metric for Domain Adaptation [CVPR2021]
  • DRANet: Disentangling Representation and Adaptation Networks for Unsupervised Cross-Domain Adaptation [CVPR2021]
  • Visualizing Adapted Knowledge in Domain Transfer [CVPR2021] [Pytorch]
  • Instance Level Affinity-Based Transfer for Unsupervised Domain Adaptation [CVPR2021] [code coming soon]
  • Dynamic Domain Adaptation for Efficient Inference [CVPR2021] [Pytorch]
  • Transferable Semantic Augmentation for Domain Adaptation [CVPR2021] [Pytorch]
  • MetaAlign: Coordinating Domain Alignment and Classification for Unsupervised Domain Adaptation [CVPR2021]
  • DRANet: Disentangling Representation and Adaptation Networks for Unsupervised Cross-Domain Adaptation [CVPR2021]
  • Dynamic Weighted Learning for Unsupervised Domain Adaptation [CVPR2021]
  • Domain Adaptation with Conditional Distribution Matching and Generalized Label Shift [NeurIPS 2020]
  • Transferable Calibration with Lower Bias and Variance in Domain Adaptation [NeurIPS 2020]
  • A Dictionary Approach to Domain-Invariant Learning in Deep Networks [NeurIPS 2020]
  • Heuristic Domain Adaptation [NeurIPS2020] [Pytorch]
  • Unsupervised Domain Adaptation for Semantic Segmentation of NIR Images through Generative Latent Search [ECCV2020][code]
  • Mind the Discriminability: Asymmetric Adversarial Domain Adaptation [ECCV2020]
  • Domain2Vec: Domain Embedding for Unsupervised Domain Adaptation [ECCV2020]
  • CSCL: Critical Semantic-Consistent Learning for Unsupervised Domain Adaptation [ECCV2020]
  • Minimum Class Confusion for Versatile Domain Adaptation [ECCV2020]
  • Partially-Shared Variational Auto-encoders for Unsupervised Domain Adaptation with Target Shift [ECCV2020] [Pytorch]
  • Label Propagation with Augmented Anchors: A Simple Semi-Supervised Learning baseline for Unsupervised Domain Adaptation [ECCV2020] [PyTorch]
  • Unsupervised Domain Adaptation via Structurally Regularized Deep Clustering [CVPR2020 Oral] [Pytorch]
  • Towards Discriminability and Diversity: Batch Nuclear-norm Maximization under Label Insufficient Situations [CVPR2020 Oral] [Pytorch]
  • Unsupervised Domain Adaptation With Hierarchical Gradient Synchronization [CVPR2020]
  • Spherical Space Domain Adaptation With Robust Pseudo-Label Loss [CVPR2020] [Pytorch]
  • Stochastic Classifiers for Unsupervised Domain Adaptation [CVPR2020]
  • Structure Preserving Generative Cross-Domain Learning [CVPR2020]
  • Light-weight Calibrator: A Separable Component for Unsupervised Domain Adaptation [CVPR2020] [code]
  • Domain Adaptive Multiflow Networks [ICLR2020]
  • Unsupervised Domain Adaptation via Discriminative Manifold Embedding and Alignment [AAAI2020]
  • Visual Domain Adaptation by Consensus-based Transfer to Intermediate Domain [Paper]
  • Unsupervised Domain Adaptation via Structured Prediction Based Selective Pseudo-Labeling [AAAI2020] [Matlab]
  • CUDA: Contradistinguisher for Unsupervised Domain Adaptation [ICDM2019]
  • Domain Adaptation with Asymmetrically-Relaxed Distribution Alignment [ICML2019]
  • Batch Weight for Domain Adaptation With Mass Shift [ICCV2019]
  • Switchable Whitening for Deep Representation Learning [ICCV2019] [pytorch]
  • Confidence Regularized Self-Training [ICCV2019 Oral] [Pytorch]
  • Larger Norm More Transferable: An Adaptive Feature Norm Approach for Unsupervised Domain Adaptation [ICCV2019] [Pytorch(official)]
  • Transferrable Prototypical Networks for Unsupervised Domain Adaptation [CVPR2019(Oral)]
  • Sliced Wasserstein Discrepancy for Unsupervised Domain Adaptation [CVPR2019]
  • Unsupervised Domain Adaptation using Feature-Whitening and Consensus Loss [CVPR 2019] [Pytorch]
  • Domain Specific Batch Normalization for Unsupervised Domain Adaptation [CVPR2019] [Pytorch]
  • AdaGraph: Unifying Predictive and Continuous Domain Adaptation through Graphs [CVPR2019] [Pytorch]
  • Unsupervised Visual Domain Adaptation: A Deep Max-Margin Gaussian Process Approach [CVPR2019] [Project]
  • Contrastive Adaptation Network for Unsupervised Domain Adaptation [CVPR2019] [Pytorch]
  • Distant Supervised Centroid Shift: A Simple and Efficient Approach to Visual Domain Adaptation [CVPR2019]
  • Unsupervised Domain Adaptation via Calibrating Uncertainties [CVPRW2019]
  • Bayesian Uncertainty Matching for Unsupervised Domain Adaptation [IJCAI2019]
  • Unsupervised Domain Adaptation for Distance Metric Learning [ICLR2019]
  • Co-regularized Alignment for Unsupervised Domain Adaptation [NIPS2018]
  • Domain Invariant and Class Discriminative Feature Learning for Visual Domain Adaptation [TIP 2018]
  • Graph Adaptive Knowledge Transfer for Unsupervised Domain Adaptation [ECCV2018]
  • Aligning Infinite-Dimensional Covariance Matrices in Reproducing Kernel Hilbert Spaces for Domain Adaptation [CVPR2018]
  • Unsupervised Domain Adaptation with Distribution Matching Machines [AAAI2018]
  • Learning to cluster in order to transfer across domains and tasks [ICLR2018] [Bolg] [Pytorch]
  • Self-Ensembling for Visual Domain Adaptation [ICLR2018]
  • Minimal-Entropy Correlation Alignment for Unsupervised Deep Domain Adaptation [ICLR2018] [TensorFlow]
  • Associative Domain Adaptation [ICCV2017] [TensorFlow] [Pytorch]
  • AutoDIAL: Automatic DomaIn Alignment Layers [ICCV2017]
  • Asymmetric Tri-training for Unsupervised Domain Adaptation [ICML2017] [TensorFlow]
  • Learning Transferrable Representations for Unsupervised Domain Adaptation [NIPS2016]

Journal

Arxiv

Foundation-Models based DA

Conference

  • POUF: Prompt-oriented unsupervised fine-tuning for large pre-trained models [ICML2023] [Pytorch]

Semi-supervised DA

Conference

  • Semi-Supervised Domain Adaptation With Source Label Adaptation [CVPR 2023]
  • Multi-level Consistency Learning for Semi-supervised Domain Adaptation [IJCAI 2022]
  • AdaMatch: A Unified Approach to Semi-Supervised Learning and Domain Adaptation [ICLR 2022]
  • CLDA: Contrastive Learning for Semi-Supervised Domain Adaptation [NeurIPS]
  • Deep Co-Training With Task Decomposition for Semi-Supervised Domain Adaptation [ICCV2021]
  • ECACL: A Holistic Framework for Semi-Supervised Domain Adaptation [ICCV2021]
  • Cross-Domain Adaptive Clustering for Semi-Supervised Domain Adaptation [CVPR2021]
  • Semi-supervised Domain Adaptation based on Dual-level Domain Mixing for Semantic Segmentation [CVPR2021]
  • Learning Invariant Representations and Risks for Semi-supervised Domain Adaptation [CVPR2021]
  • Improving Semi-Supervised Domain Adaptation Using Effective Target Selection and Semantics [CVPRW2021] [Code]
  • Attract, Perturb, and Explore: Learning a Feature Alignment Network for Semi-supervised Domain Adaptation [ECCV2020]
  • Online Meta-Learning for Multi-Source and Semi-Supervised Domain Adaptation [ECCV2020]
  • Bidirectional Adversarial Training for Semi-Supervised Domain Adaptation [IJCAI2020]
  • Semi-supervised Domain Adaptation via Minimax Entropy [ICCV2019] [Pytorch]

Journal

Arxiv

  • Pred&Guide: Labeled Target Class Prediction for Guiding Semi-Supervised Domain Adaptation [22 Nov 2022]
  • MiCo: Mixup Co-Training for Semi-Supervised Domain Adaptation [ 24 Jul 2020]
  • Opposite Structure Learning for Semi-supervised Domain Adaptation [6 Feb 2020]
  • Reducing Domain Gap via Style-Agnostic Networks [25 Oct 2019]

Weakly-Supervised DA

Conference

  • Towards Accurate and Robust Domain Adaptation under Noisy Environments [IJCAI2020]
  • Weakly Supervised Open-set Domain Adaptation by Dual-domain Collaboration [CVPR2019]
  • Transferable Curriculum for Weakly-Supervised Domain Adaptation [AAAI2019]

Arxiv

Zero-shot DA

Conference

  • Collaborative Learning With Disentangled Features for Zero-Shot Domain Adaptation [ICCV2021]
  • Zero-Shot Day-Night Domain Adaptation with a Physics Prior [ICCV2021]
  • High Resolution Zero-Shot Domain Adaptation of Synthetically Rendered Face Images [ECCV2020]
  • Adversarial Learning for Zero-shot Domain Adaptation [ECCV2020]
  • HGNet: Hybrid Generative Network for Zero-shot Domain Adaptation [ECCV2020]
  • Zero-shot Domain Adaptation Based on Attribute Information [ACML2019]
  • Conditional Coupled Generative Adversarial Networks for Zero-Shot Domain Adaptation [ICCV2019]
  • Generalized Zero-Shot Learning with Deep Calibration Network [NIPS2018]
  • Zero-Shot Deep Domain Adaptation [ECCV2018]

One-shot DA

Conference

Arxiv

  • One-Shot Imitation from Observing Humans via Domain-Adaptive Meta-Learning [arxiv]

Few-shot UDA

Conference

Arxiv

  • Cross-domain Self-supervised Learning for Domain Adaptation with Few Source Labels [arXiv 18 Mar 2020]

Few-shot DA

Conference

Arxiv

  • Feature transformation ensemble model with batch spectral regularization for cross-domain few-shot classification [arXiv 18 May 2020] [Pytorch]
  • Ensemble model with batch spectral regularization and data blending for cross-domain few-shot learning with unlabeled data [arXiv 8 June 2020] [Pytorch]

Partial DA

Conference

Journal

  • Unsupervised Multi-Class Domain Adaptation: Theory, Algorithms, and Practice [TPAMI2020] [PyTroch]

Arxiv

  • Select, Label, and Mix: Learning Discriminative Invariant Feature Representations for Partial Domain Adaptation [arXiv 06 Dec 2020]
  • Unsupervised Multi-Class Domain Adaptation: Theory, Algorithms, and Practice [20 Feb 2020] [PyTroch]
  • Tackling Partial Domain Adaptation with Self-Supervision [arXiv 12 Jun 2019]
  • Domain Adversarial Reinforcement Learning for Partial Domain Adaptation [arXiv 10 May 2019]

Open Set DA

Conference

Journal

Arxiv

  • Collaborative Training of Balanced Random Forests for Open Set Domain Adaptation [10 Feb 2020]
  • Known-class Aware Self-ensemble for Open Set Domain Adaptation [3 May 2019]

Universal DA

Conference

Journal

Arxiv

  • Universal Multi-Source Domain Adaptation [5 Nov 2020]
  • A Sample Selection Approach for Universal Domain Adaptation [14 Jan 2020]

Open Compound DA

Conference

Journal

  • Source-Free Open Compound Domain Adaptation in Semantic Segmentation [TCSVT 2022]

Multi Source DA

Conference

  • Confident Anchor-Induced Multi-Source Free Domain Adaptation [NeurIPS2021] [code is coming soon]
  • mDALU: Multi-Source Domain Adaptation and Label Unification With Partial Datasets [ICCV2021]
  • STEM: An Approach to Multi-Source Domain Adaptation With Guarantees [ICCV2021]
  • T-SVDNet: Exploring High-Order Prototypical Correlations for Multi-Source Domain Adaptation [ICCV2021]
  • Multi-Source Domain Adaptation for Object Detection [ICCV2021]
  • Information-Theoretic Regularization for Multi-Source Domain Adaptation [ICCV2021]
  • Partial Feature Selection and Alignment for Multi-Source Domain Adaptation [CVPR2021]
  • Wasserstein Barycenter for Multi-Source Domain Adaptation [CVPR2021] [Code]
  • Unsupervised Multi-source Domain Adaptation Without Access to Source Data [CVPR2021]
  • Dynamic Transfer for Multi-Source Domain Adaptation [CVPR2021] [Pytorch]
  • Multi-Source Domain Adaptation with Collaborative Learning for Semantic Segmentation [CVPR2021]
  • MOST: Multi-Source Domain Adaptation via Optimal Transport for Student-Teacher Learning [UAI2021]
  • Meta Self-Learning for Multi-Source Domain Adaptation: A Benchmark [ICCV Workshop 2021] [Pytorch]
  • Your Classifier can Secretly Suffice Multi-Source Domain Adaptation [NeurIPS 2020] [Project]
  • Multi-Source Open-Set Deep Adversarial Domain Adaptation [ECCV2020]
  • Online Meta-Learning for Multi-Source and Semi-Supervised Domain Adaptation [ECCV2020]
  • Multi-Source Open-Set Deep Adversarial Domain Adaptation [ECCV2020]
  • Curriculum Manager for Source Selection in Multi-Source Domain Adaptation [ECCV2020]
  • Domain Aggregation Networks for Multi-Source Domain Adaptation [ICML2020]
  • Learning to Combine: Knowledge Aggregation for Multi-Source Domain Adaptation [ECCV2020] [Pytorch]
  • Multi-Source Domain Adaptation for Text Classification via DistanceNet-Bandits [AAAI2020]
  • Adversarial Training Based Multi-Source Unsupervised Domain Adaptation for Sentiment Analysis [AAAI2020]
  • Multi-source Domain Adaptation for Visual Sentiment Classification [AAAI2020]
  • Multi-source Distilling Domain Adaptation [AAAI2020] [code]
  • Multi-source Domain Adaptation for Semantic Segmentation [NeurlPS2019] [Pytorch]
  • Moment Matching for Multi-Source Domain Adaptation [ICCV2019] [Pytorch]
  • Multi-Domain Adversarial Learning [ICLR2019] [Torch]
  • Algorithms and Theory for Multiple-Source Adaptation [NIPS2018]
  • Adversarial Multiple Source Domain Adaptation [NIPS2018] [Pytorch]
  • Boosting Domain Adaptation by Discovering Latent Domains [CVPR2018] [Caffe] [Pytorch]
  • Deep Cocktail Network: Multi-source Unsupervised Domain Adaptation with Category Shift [CVPR2018] [Pytorch]

Journal

Arxiv

  • Mutual learning network for multi-source domain adaptation [arXiv]
  • Domain Adaptive Ensemble Learning [arXiv]
  • Multi-Source Domain Adaptation and Semi-Supervised Domain Adaptation with Focus on Visual Domain Adaptation Challenge 2019 [14 Oct 2019]

Multi Target DA

Conference

  • CoNMix for Source-free Single and Multi-target Domain Adaptation [WACV2022] [Pytorch]
  • Curriculum Graph Co-Teaching for Multi-Target Domain Adaptation [CVPR2021] [Pytorch]
  • Multi-Target Domain Adaptation with Collaborative Consistency Learning [CVPR2021]

Arxiv

  • Unsupervised Multi-Target Domain Adaptation: An Information Theoretic Approach [arXiv]

Incremental DA

Conference

  • Lifelong Domain Adaptation via Consolidated Internal Distribution [NeurIPS2021]
  • Continual Adaptation of Visual Representations via Domain Randomization and Meta-learning [CVPR2021]
  • ConDA: Continual Unsupervised Domain Adaptation [CVPR2021]
  • Gradient Regularized Contrastive Learning for Continual Domain Adaptation [AAAI2021]
  • Gradual Domain Adaptation without Indexed Intermediate Domains [NeurIPS2021]
  • Learning to Adapt to Evolving Domains [NeurIPS 2020] [Pytorch]
  • Class-Incremental Domain Adaptation [ECCV2020]
  • Incremental Adversarial Domain Adaptation for Continually Changing Environments [ICRA2018]
  • Continuous Manifold based Adaptation for Evolving Visual Domains [CVPR2014]

Multi Step DA

Arxiv

  • Adversarial Domain Adaptation for Stance Detection [arXiv]
  • Ensemble Adversarial Training: Attacks and Defenses [arXiv]

Conference

Heterogeneous DA

Conference

  • Domain Adaptive Classification on Heterogeneous Information Networks [IJCAI2020]
  • Heterogeneous Domain Adaptation via Soft Transfer Network [ACM MM2019]

Target-agnostic DA

Arxiv

Conference

Federated DA

Arxiv

Continuously Indexed DA

Conference

Source Free DA

Conference

Arxiv

  • Learning Invariant Representation with Consistency and Diversity for Semi-supervised Source Hypothesis Transfer[7 Jul 2021][Pytorch]
  • Source Data-absent Unsupervised Domain Adaptation through Hypothesis Transfer and Labeling Transfer [14 Dec 2020] [Pytorch]

Active DA

Conference

  • Local Context-Aware Active Domain Adaptation [ICCV2023] [Pytorch]
  • Reducing Annotation Effort by Identifying and Labeling Contextually Diverse Classes for Semantic Segmentation Under Domain Shift [WACV2023]
  • Towards Fewer Annotations: Active Learning via Region Impurity and Prediction Uncertainty for Domain Adaptive Semantic Segmentation [CVPR2022][Pytorch]
  • Active Learning for Domain Adaptation: An Energy-based Approach [AAAI2022][Pytorch]
  • Multi-Anchor Active Domain Adaptation for Semantic Segmentation [ICCV2021]
  • Active Domain Adaptation via Clustering Uncertainty-Weighted Embeddings [ICCV2021]
  • Active Universal Domain Adaptation [ICCV2021]
  • S3VAADA: Submodular Subset Selection for Virtual Adversarial Active Domain Adaptation [ICCV2021]
  • Transferable Query Selection for Active Domain Adaptation [CVPR2021]

Generalized Domain Adaptation

Conference

Model Selection

Other Transfer Learning Paradigms

Domain Generalization

Conference

Journal

Arxiv

  • Adversarial Pyramid Network for Video Domain Generalization [8 Dec 2019]
  • Towards Shape Biased Unsupervised Representation Learning for Domain Generalization [18 Sep 2019]
  • A Generalization Error Bound for Multi-class Domain Generalization [24 May 2019]
  • Adversarial Invariant Feature Learning with Accuracy Constraint for Domain Generalization [29 Apr 2019]
  • Beyond Domain Adaptation: Unseen Domain Encapsulation via Universal Non-volume Preserving Models [9 Dec 2018]

Domain Randomization

Conference

  • DeceptionNet: Network-Driven Domain Randomization [ICCV2019]
  • Domain Randomization and Pyramid Consistency: Simulation-to-Real Generalization Without Accessing Target Domain Data [ICCV2019]

Transfer Metric Learning

  • Transfer Metric Learning: Algorithms, Applications and Outlooks [arXiv]

Knowledge Transfer

Conference

  • Attention Bridging Network for Knowledge Transfer [ICCV2019]
  • Few-Shot Image Recognition with Knowledge Transfer [ICCV2019]

Others

Conference

  • Learning Across Tasks and Domains [ICCV2019]
  • UM-Adapt: Unsupervised Multi-Task Adaptation Using Adversarial Cross-Task Distillation [ICCV2019]
  • Domain Agnostic Learning with Disentangled Representations [ICML2019]
  • Unsupervised Open Domain Recognition by Semantic Discrepancy Minimization [CVPR2019] [Pytorch]

Arxiv

  • GradMix: Multi-source Transfer across Domains and Tasks [[9 Feb 2020]](GradMix: Multi-source Transfer across Domains and Tasks)
  • When Semi-Supervised Learning Meets Transfer Learning: Training Strategies, Models and Datasets [arXiv 13 Dec 2018]

Applications

Object Detection

Survey

Conference

  • Supervision Interpolation via LossMix: Generalizing Mixup for Object Detection and Beyond [AAAI2024]
  • Instance Relation Graph Guided Source-Free Domain Adaptive Object Detection [CVPR2023] [Project]
  • Towards Online Domain Adaptive Object Detection [WACV2023] [[https://github.com/Vibashan/online-da]]
  • Mixture of Teacher Experts for Source-Free Domain Adaptive Object Detection [ICIP2022]
  • Towards Robust Adaptive Object Detection under Noisy Annotations [CVPR2022] [PyTorch]
  • H2FA R-CNN: Holistic and Hierarchical Feature Alignment for Cross-Domain Weakly Supervised Object Detection [CVPR2022] [PyTorch] [PaddlePaddle]
  • Cross-Domain Adaptive Teacher for Object Detection [CVPR2022] [Project] [PyTorch]
  • Task-specific Inconsistency Alignment for Domain Adaptive Object Detection [CVPR2022] [PyTorch]
  • SIGMA: Semantic-complete Graph Matching for Domain Adaptive Object Detection [CVPR2022] [PyTorch]
  • Single-Domain Generalized Object Detection in Urban Scene via Cyclic-Disentangled Self-Distillation [CVPR2022]
  • Target-Relevant Knowledge Preservation for Multi-Source Domain Adaptive Object Detection [CVPR2022]
  • Cross Domain Object Detection by Target-Perceived Dual Branch Distillation [CVPR2022]
  • Decoupled Adaptation for Cross-Domain Object Detection [ICLR2022] [PyTorch]
  • SCAN: Cross Domain Object Detection with Semantic Conditioned Adaptation [AAAI2022] [PyTorch]
  • SSAL: Synergizing between Self-Training and Adversarial Learning for Domain Adaptive Object Detection [NeurIPS2021] [Project]
  • Multi-Source Domain Adaptation for Object Detection [ICCV2021]
  • Knowledge Mining and Transferring for Domain Adaptive Object Detection [ICCV2021]
  • Dual Bipartite Graph Learning: A General Approach for Domain Adaptive Object Detection [ICCV2021]
  • Seeking Similarities over Differences: Similarity-based Domain Alignment for Adaptive Object Detection [ICCV2021]
  • Informative and Consistent Correspondence Mining for Cross-Domain Weakly Supervised Object Detection [CVPR2021]
  • MeGA-CDA: Memory Guided Attention for Category-Aware Unsupervised Domain Adaptive Object Detection [CVPR2021]
  • SRDAN: Scale-aware and Range-aware Domain Adaptation Network
    for Cross-dataset 3D Object Detection [CVPR2021]
  • I3Net: Implicit Instance-Invariant Network for Adapting One-Stage Object Detectors [CVPR2021]
  • RPN Prototype Alignment for Domain Adaptive Object Detector [CVPR2021]
  • ST3D: Self-training for Unsupervised Domain Adaptation on 3D ObjectDetection [CVPR2021]
  • Domain-Specific Suppression for Adaptive Object Detection [CVPR2021]
  • Unbiased Mean Teacher for Cross-Domain Object Detection [CVPR2021]
  • YOLO in the Dark - Domain Adaptation Method for Merging Multiple Models [ECCV2020]
  • Collaborative Training between Region Proposal Localization and Classification for Domain Adaptive Object Detection [ECCV2020]
  • One-Shot Unsupervised Cross-Domain Detection [ECCV2020]
  • Every Pixel Matters: Center-aware Feature Alignment for Domain Adaptive Object Detector [ECCV2020]
  • Adapting Object Detectors with Conditional Domain Normalization [ECCV2020]
  • Prior-based Domain Adaptive Object Detection for Hazy and Rainy Conditions [ECCV2020]
  • Domain Adaptive Object Detection via Asymmetric Tri-way Faster-RCNN [ECCV2020]
  • Cross-domain Object Detection through Coarse-to-Fine Feature Adaptation [CVPR2020]
  • Harmonizing Transferability and Discriminability for Adapting Object Detectors [CVPR2020] [code]
  • Exploring Categorical Regularization for Domain Adaptive Object Detection [CVPR2020] [code]
  • Cross-domain Detection via Graph-induced Prototype Alignment [CVPR2020 Oral] [code]
  • Multi-spectral Salient Object Detection by Adversarial Domain Adaptation [Paper]
  • Deep Domain Adaptive Object Detection: a Survey [ICIP2020]
  • Progressive Domain Adaptation for Object Detection [WACV]
  • Cross-Domain Car Detection Using Unsupervised Image-to-Image Translation: From Day to Night [IJCNN2019 Oral] [Project]
  • Self-Training and Adversarial Background Regularization for Unsupervised Domain Adaptive One-Stage Object Detection [ICCV2019 Oral]
  • A Robust Learning Approach to Domain Adaptive Object Detection [ICCV2019] [code]
  • Multi-adversarial Faster-RCNN for Unrestricted Object Detection [ICCV2019]
  • Few-Shot Adaptive Faster R-CNN [CVPR2019]
  • Exploring Object Relation in Mean Teacher for Cross-Domain Detection [CVPR2019]
  • Adapting Object Detectors via Selective Cross-Domain Alignment [CVPR2019] [Pytorch]
  • Automatic adaptation of object detectors to new domains using self-training [CVPR2019] [Project]
  • Towards Universal Object Detection by Domain Attention [CVPR2019]
  • Strong-Weak Distribution Alignment for Adaptive Object Detection [CVPR2019] [Pytorch]
  • Diversify and Match: A Domain Adaptive Representation Learning Paradigm for Object Detection [CVPR2019] [Pytorch]
  • Cross-Domain Weakly-Supervised Object Detection Through Progressive Domain Adaptation [CVPR2018]
  • Domain Adaptive Faster R-CNN for Object Detection in the Wild [CVPR2018] [Caffe2] [Caffe] [Pytorch(under developing)]

Journal

  • Cross-domain object detection using unsupervised image translation [ESWA]
  • Pixel and feature level based domain adaptation for object detection in autonomous driving [Neurocomputing]

Arxiv

  • See Eye to Eye: A Lidar-Agnostic 3D Detection Framework for Unsupervised Multi-Target Domain Adaptation [17 Nov 2021]
  • Unsupervised Domain Adaptive Object Detection using Forward-Backward Cyclic Adaptation [3 Feb 2020]
  • Prior-based Domain Adaptive Object Detection for Adverse Weather Conditions [29 Nov 2019]
  • Unsupervised Domain Adaptation for Object Detection via Cross-Domain Semi-Supervised Learning [17 Nov 2019]
  • Curriculum Self-Paced Learning for Cross-Domain Object Detection [15 Nov 2019]
  • SCL: Towards Accurate Domain Adaptive Object Detection via Gradient Detach Based Stacked Complementary Losses [6 Nov 2019]

Semantic Segmentation

Conference

  • PiPa: Pixel- and Patch-wise Self-supervised Learning for Domain Adaptative Semantic Segmentation [ACM MM2023] [Pytorch]
  • Refign: Align and Refine for Adaptation of Semantic Segmentation to Adverse Conditions [WACV 2023] [Pytorch]
  • Deliberated Domain Bridging for Domain Adaptive Semantic Segmentation [NeruIPS 2022] [Pytorch]
  • DecoupleNet: Decoupled Network for Domain Adaptive Semantic Segmentation [ECCV 2022] [Pytorch]
  • HRDA: Context-Aware High-Resolution Domain-Adaptive Semantic Segmentation [ECCV 2022] [Pytorch]
  • Style-Hallucinated Dual Consistency Learning for Domain Generalized Semantic Segmentation [ECCV 2022] [Pytorch]
  • DAFormer: Improving Network Architectures and Training Strategies for Domain-Adaptive Semantic Segmentation [CVPR 2022] [Pytorch]
  • Plugging Self-Supervised Monocular Depth into Unsupervised Domain Adaptation for Semantic Segmentation [WACV 2022]
  • Shallow Features Guide Unsupervised Domain Adaptation for Semantic Segmentation at Class Boundaries [WACV 2022]
  • Learning to Adapt via Latent Domains for Adaptive Semantic Segmentation [NeurIPS2021]
  • Dual Path Learning for Domain Adaptation of Semantic Segmentation [ICCV2021]
  • Exploring Robustness of Unsupervised Domain Adaptation in Semantic Segmentation [ICCV2021]
  • Multi-Anchor Active Domain Adaptation for Semantic Segmentation [ICCV2021]
  • LabOR: Labeling Only if Required for Domain Adaptive Semantic Segmentation [ICCV2021]
  • Self-Mutating Network for Domain Adaptive Segmentation in Aerial Images [ICCV2021]
  • Geometric Unsupervised Domain Adaptation for Semantic Segmentation [ICCV2021]
  • Multi-Target Adversarial Frameworks for Domain Adaptation in Semantic Segmentation [ICCV2021]
  • BAPA-Net: Boundary Adaptation and Prototype Alignment for Cross-Domain Semantic Segmentation [ICCV2021]
  • BiMaL: Bijective Maximum Likelihood Approach to Domain Adaptation in Semantic Scene Segmentation [ICCV2021]
  • Uncertainty-Aware Pseudo Label Refinery for Domain Adaptive Semantic Segmentation [ICCV2021]
  • Domain Adaptive Semantic Segmentation With Self-Supervised Depth Estimation [ICCV2021]
  • Generalize Then Adapt: Source-Free Domain Adaptive Semantic Segmentation [ICCV2021]
  • DARCNN: Domain Adaptive Region-Based Convolutional Neural Network for Unsupervised Instance Segmentation in Biomedical Images [CVPR2021]
  • DANNet: A One-Stage Domain Adaptation Network for Unsupervised Nighttime Semantic Segmentation [CVPR2021]
  • Prototypical Pseudo Label Denoising and Target Structure Learning for Domain Adaptive Semantic Segmentation [CVPR2021]
  • Complete & Label: A Domain Adaptation Approach to Semantic Segmentation of LiDAR Point Clouds [CVPR2021]
  • Cluster, Split, Fuse, and Update: Meta-Learning for Open Compound Domain Adaptive Semantic Segmentation [CVPR2021]
  • PixMatch: Unsupervised Domain Adaptation via Pixelwise Consistency Training [CVPR2021] [Pytorch]
  • Learning to Relate Depth and Semantics for Unsupervised Domain Adaptation [CVPR2021] [Pytorch]
  • Cross-View Regularization for Domain Adaptive Panoptic Segmentation [CVPR2021]
  • Semi-supervised Domain Adaptation based on Dual-level Domain Mixing for Semantic Segmentation [CVPR2021]
  • MetaCorrection: Domain-aware Meta Loss Correction for Unsupervised Domain Adaptation in Semantic Segmentation [CVPR2021]
  • Coarse-to-Fine Domain Adaptive Semantic Segmentation with Photometric Alignment and Category-Center Regularization [CVPR2021]
  • Multi-Source Domain Adaptation with Collaborative Learning for Semantic Segmentation [CVPR2021]
  • Source-Free Domain Adaptation for Semantic Segmentation [CVPR2021]
  • Instance Adaptive Self-Training for Unsupervised Domain Adaptation [ECCV 2020] [Pytorch]
  • Cross-stained Segmentation from Renal Biopsy Images Using Multi-level Adversarial Learning [ICASSP 2020]
  • Pixel-Level Cycle Association: A New Perspective for Domain Adaptive Semantic Segmentation [NeurlIPS 2020] [Pytorch]
  • Adversarial Style Mining for One-Shot Unsupervised Domain Adaptation [NeurIPS2020] [Pytorch]
  • Semantically Adaptive Image-to-image Translation for Domain Adaptation of Semantic Segmentation [BMVC2020]
  • Contextual-Relation Consistent Domain Adaptation for Semantic Segmentation [ECCV2020]
  • Learning from Scale-Invariant Examples for Domain Adaptation in Semantic Segmentation [ECCV2020]
  • Label-Driven Reconstruction for Domain Adaptation in Semantic Segmentation [ECCV2020]
  • Unsupervised Domain Adaptation for Semantic Segmentation of NIR Images through Generative Latent Search [ECCV2020]
  • Domain Adaptive Semantic Segmentation Using Weak Labels [ECCV2020]
  • Content-Consistent Matching for Domain Adaptive Semantic Segmentation [ECCV2020] [PyTorch]
  • Cross-Domain Semantic Segmentation via Domain-Invariant Interactive Relation Transfer [CVPR2020]
  • Phase Consistent Ecological Domain Adaptation [CVPR2020] [Pytorch]
  • FDA: Fourier Domain Adaptation for Semantic Segmentation [CVPR2020] [Pytorch]
  • Unsupervised Instance Segmentation in Microscopy Images via Panoptic Domain Adaptation and Task Re-weighting [CVPR2020]
  • Unsupervised Intra-domain Adaptation for Semantic Segmentation through Self-Supervision [CVPR2020 Oral] [Pytorch]
  • Differential Treatment for Stuff and Things: A Simple Unsupervised Domain Adaptation Method for Semantic Segmentation [CVPR2020]
  • Learning Texture Invariant Representation for Domain Adaptation of Semantic Segmentation [CVPR2020] [Pytorch]
  • xMUDA: Cross-Modal Unsupervised Domain Adaptation for 3D Semantic Segmentation [CVPR2020] [Demo] [code]
  • Unsupervised Scene Adaptation with Memory Regularization in vivo [IJCAI2020] [code]
  • Joint Adversarial Learning for Domain Adaptation in Semantic Segmentation [AAAI2020]
  • An Adversarial Perturbation Oriented Domain Adaptation Approach for Semantic Segmentation [AAAI2020]
  • Category Anchor-Guided Unsupervised Domain Adaptation for Semantic Segmentation [NeurIPS2019] [code]
  • MLSL: Multi-Level Self-Supervised Learning for Domain Adaptation with Spatially Independent and Semantically Consistent Labeling [WACV2020]
  • Domain Bridge for Unpaired Image-to-Image Translation and Unsupervised Domain Adaptation [WACV2020]
  • Guided Curriculum Model Adaptation and Uncertainty-Aware Evaluation for
    Semantic Nighttime Image Segmentation [ICCV2019]
  • Constructing Self-motivated Pyramid Curriculums for Cross-Domain Semantic
    Segmentation: A Non-Adversarial Approach [ICCV2019] [Pytorch]
  • SSF-DAN: Separated Semantic Feature Based Domain Adaptation Network for Semantic Segmentation [ICCV2019]
  • Significance-aware Information Bottleneck for Domain Adaptive Semantic Segmentation [ICCV2019]
  • Domain Adaptation for Semantic Segmentation with Maximum Squares Loss [ICCV2019] [Pytorch]
  • Self-Ensembling with GAN-based Data Augmentation for Domain Adaptation in Semantic Segmentation [ICCV2019]
  • DADA: Depth-aware Domain Adaptation in Semantic Segmentation [ICCV2019] [code]
  • Domain Adaptation for Structured Output via Discriminative Patch Representations [ICCV2019 Oral] [Project]
  • Not All Areas Are Equal: Transfer Learning for Semantic Segmentation via Hierarchical Region Selection [CVPR2019(Oral)]
  • CrDoCo: Pixel-level Domain Transfer with Cross-Domain Consistency [CVPR2019] [Project] [Pytorch]
  • Bidirectional Learning for Domain Adaptation of Semantic Segmentation [CVPR2019] [Pytorch]
  • Learning Semantic Segmentation from Synthetic Data: A Geometrically Guided Input-Output Adaptation Approach [CVPR2019]
  • All about Structure: Adapting Structural Information across Domains for Boosting Semantic Segmentation [CVPR2019] [Pytorch]
  • DLOW: Domain Flow for Adaptation and Generalization [CVPR2019 Oral]
  • Taking A Closer Look at Domain Shift: Category-level Adversaries for Semantics Consistent Domain Adaptation [CVPR2019 Oral] [Pytorch]
  • ADVENT: Adversarial Entropy Minimization for Domain Adaptation in Semantic Segmentation [CVPR2019 Oral] [Pytorch]
  • SPIGAN: Privileged Adversarial Learning from Simulation [ICLR2019]
  • Penalizing Top Performers: Conservative Loss for Semantic Segmentation Adaptation [ECCV2018]
  • Domain transfer through deep activation matching [ECCV2018]
  • Unsupervised Domain Adaptation for Semantic Segmentation via Class-Balanced Self-Training [ECCV2018] [Pytorch]
  • DCAN: Dual channel-wise alignment networks for unsupervised scene adaptation [ECCV2018]
  • Fully convolutional adaptation networks for semantic
    segmentation [CVPR2018]
  • Learning to Adapt Structured Output Space for Semantic Segmentation [CVPR2018] [Pytorch]
  • Conditional Generative Adversarial Network for Structured Domain Adaptation [CVPR2018]
  • Learning From Synthetic Data: Addressing Domain Shift for Semantic Segmentation [CVPR2018] [Pytorch]
  • Curriculum Domain Adaptation for Semantic Segmentation of Urban Scenes [ICCV2017] [Journal Version] [Keras]
  • No more discrimination: Cross city adaptation of road scene segmenters [ICCV2017]

Journal

  • SePiCo: Semantic-Guided Pixel Contrast for Domain Adaptive Semantic Segmentation [TPAMI2023][arxiv version][Pytorch]
  • Adaptive Boosting for Domain Adaptation: Towards Robust Predictions in Scene Segmentation [TIP2022][Pytorch]
  • Rectifying Pseudo Label Learning via Uncertainty Estimation for Domain Adaptive Semantic Segmentation [IJCV2020][Pytorch]
  • Multi-level colonoscopy malignant tissue detection with adversarial CAC-UNet [Neurocomputing 2021] [Pytorch]
  • Affinity Space Adaptation for Semantic Segmentation Across Domains [TIP2020][Pytorch]
  • Semantic-aware short path adversarial training for cross-domain semantic segmentation [Neurocomputing 2019]
  • Weakly Supervised Adversarial Domain Adaptation for Semantic Segmentation in Urban Scenes [TIP]

Arxiv

  • Class-Conditional Domain Adaptation on Semantic Segmentation [27 Nov 2019]
  • Adversarial Learning and Self-Teaching Techniques for Domain Adaptation in Semantic Segmentation [2 Sep 2019]
  • FCNs in the Wild: Pixel-level Adversarial and Constraint-based Adaptation [8 Dec 2016]
  • BoMuDA: Boundless Multi-Source Domain Adaptive Segmentation in Unconstrained Environments [13 Oct 2020][Pytorch]
  • SAfE: Self-Attention Based Unsupervised Road Safety Classification in Hazardous Environments [27 Nov 2020][Pytorch]
  • Semantics-aware Multi-modal Domain Translation:From LiDAR Point Clouds to Panoramic Color Images [26 Jun 2021] [Pytorch]

Person Re-identification

Conference

  • Learning to Generalize Unseen Domains via Memory-based Multi-Source Meta-Learning for Person Re-Identification [CVPR 2021] [Pytorch]
  • Group-aware Label Transfer for Domain Adaptive Person Re-identification [CVPR2021]
  • Unsupervised Domain Adaptation in the Dissimilarity Space for Person Re-identification [ECCV2020]
  • Joint Visual and Temporal Consistency for Unsupervised Domain Adaptive Person Re-Identification [ECCV2020]
  • Joint Disentangling and Adaptation for Cross-Domain Person Re-Identification [ECV2020]
  • Multiple Expert Brainstorming for Domain Adaptive Person Re-identification [ECCV2020]
  • Deep Credible Metric Learning for Unsupervised Domain Adaptation Person Re-identification [ECCV2020]
  • Unsupervised Domain Adaptation with Noise Resistible Mutual-Training for Person Re-identification [ECCV2020]
  • Generalizing Person Re-Identification by Camera-Aware Invariance Learning and Cross-Domain Mixup [ECCV2020]
  • AD-Cluster: Augmented Discriminative Clustering for Domain Adaptive Person Re-identification [CVPR2020]
  • Smoothing Adversarial Domain Attack and P-Memory Reconsolidation for Cross-Domain Person Re-Identification [CVPR2020]
  • Cross-Modal Cross-Domain Moment Alignment Network for Person Search [CVPR2020]
  • Online Joint Multi-Metric Adaptation From Frequent Sharing-Subset Mining for Person Re-Identification [CVPR2020]
  • Mutual Mean-Teaching: Pseudo Label Refinery for Unsupervised Domain Adaptation on Person Re-identification [ICLR2020] [Pytorch]
  • Self-similarity Grouping: A Simple Unsupervised Cross Domain Adaptation Approach for Person Re-identification [ICCV2019 Oral] [Pytorch]
  • A Novel Unsupervised Camera-aware Domain Adaptation Framework for Person Re-identification [ICCV2019]
  • Invariance Matters: Exemplar Memory for Domain Adaptive Person Re-identification [CVPR2019] [Pytorch]
  • Domain Adaptation through Synthesis for Unsupervised Person Re-identification [ECCV2018]
  • Person Transfer GAN to Bridge Domain Gap for Person Re-Identification [CVPR2018]
  • Image-Image Domain Adaptation with Preserved Self-Similarity and Domain-Dissimilarity for Person Re-identification [CVPR2018]

Arxiv

Sim-to-Real Transfer

Conference

  • DIRL: Domain-Invariant Reperesentation Learning Approach for Sim-to-Real Transfer [CoRL2020] [Project]

Video Domain Adaptation

Conference

Arxiv

Conference

  • PopGenAdapt: Semi-Supervised Domain Adaptation for Genotype-to-Phenotype Prediction in Underrepresented Populations [PSB 2024]
  • Cross-stained Segmentation from Renal Biopsy Images Using Multi-level Adversarial Learning [ICASSP 2020]
  • What Can Be Transferred: Unsupervised Domain Adaptation for Endoscopic Lesions Segmentation [Paper]
  • Semantic-Transferable Weakly-Supervised Endoscopic Lesions Segmentation [ICCV2019]

Journal

Arxiv

  • On-the-Fly Test-time Adaptation for Medical Image Segmentation [10 Mar 2022] [Pytorch]
  • Target and task specific source-free domain adaptive image segmentation [10 Mar 2023] [Pytorch]
  • Unsupervised Domain Adaptation via Disentangled Representations: Application to Cross-Modality Liver Segmentation [arXiv 29 Aug 2019]
  • Synergistic Image and Feature Adaptation: Towards Cross-Modality Domain Adaptation for Medical Image Segmentation [arXiv on 24 Jan 2019]
  • Unsupervised domain adaptation for medical imaging segmentation with self-ensembling [arXiv 14 Nov 2018]

Monocular Depth Estimation

  • Geometry-Aware Symmetric Domain Adaptation for Monocular Depth Estimation [CVPR2019]
  • Real-Time Monocular Depth Estimation using Synthetic Data with Domain Adaptation via Image Style Transfer [CVPR2018]

3D

Conference

  • SPG: Unsupervised Domain Adaptation for 3D Object Detection via Semantic Point Generation [ICCV2021]
  • Sparse-to-Dense Feature Matching: Intra and Inter Domain Cross-Modal Learning in Domain Adaptation for 3D Semantic Segmentation [ICCV2021]
  • Unsupervised Domain Adaptive 3D Detection With Multi-Level Consistency [ICCV2021]
  • Domain-Adaptive Single-View 3D Reconstruction [ICCV2019]

Arxiv

Fine-Grained Domain

Conference

LiDAR

Conference

  • SALUDA: Surface-based Automotive Lidar Unsupervised Domain Adaptation [3DV 2024] [Pytorch]
  • GIPSO: Geometrically Informed Propagation for Online Adaptation in 3D LiDAR Segmentation [ECCV2022] [Pytorch]
  • CoSMix: Compositional Semantic Mix for Domain Adaptation in 3D LiDAR Segmentation [ECCV2022] [Pytorch]

ArXiv

  • ConDA: Unsupervised Domain Adaptation for LiDAR Segmentation via Regularized Domain Concatenation [13 Mar 2022]

Remote Sensing

Journal

  • Open-Set Black-Box Domain Adaptation for Remote Sensing Image Scene Classification [GRSL 2023]

Others

Conference

  • RefRec: Pseudo-labels Refinement via Shape Reconstruction for Unsupervised 3D Domain Adaptation [3DV 2021 Oral]
  • Unsupervised Domain Adaptation in LiDAR Semantic Segmentation with Self-Supervision and Gated Adapters [ICRA2022]
  • RDA: Robust Domain Adaptation via Fourier Adversarial Attacking [ICCV2021]
  • Geometry-Aware Self-Training for Unsupervised Domain Adaptation on Object Point Clouds [ICCV2021]
  • Tune It the Right Way: Unsupervised Validation of Domain Adaptation via Soft Neighborhood Density [ICCV2021]
  • PIT: Position-Invariant Transform for Cross-FoV Domain Adaptation [ICCV2021]
  • Self-Supervised Domain Adaptation for Forgery Localization of JPEG Compressed Images [ICCV2021]
  • Unsupervised Real-World Super-Resolution: A Domain Adaptation Perspective [ICCV2021]
  • Adversarial Robustness for Unsupervised Domain Adaptation [ICCV2021]
  • Collaborative Optimization and Aggregation for Decentralized Domain Generalization and Adaptation [ICCV2021]
  • Adaptive Cross-Modal Prototypes for Cross-Domain Visual-Language Retrieval [CVPR2021]
  • Spatio-temporal Contrastive Domain Adaptation for Action Recognition [CVPR2021]
  • Regressive Domain Adaptation for Unsupervised Keypoint Detection [CVPR2021]
  • From Synthetic to Real: Unsupervised Domain Adaptation for Animal Pose Estimation [CVPR2021] [code coming soon]
  • Meta Self-Learning for Multi-Source Domain Adaptation: A Benchmark [ICCV Workshop 2021] [Pytorch]
  • Adapting Neural Architectures Between Domains [NeurlPS 2020]
  • Unsupervised Domain Attention Adaptation Network for Caricature Attribute Recognition [ECCV2020]
  • A Broader Study of Cross-Domain Few-Shot Learning [ECCV2020]
  • Label-Noise Robust Domain Adaptation [ICML2020]
  • Unsupervised Domain Adaptation of a Pretrained Cross-Lingual Language Model [IJCAI2020]
  • Domain Adaptation for Semantic Parsing [IJCAI2020]
  • Bridging Cross-Tasks Gap for Cognitive Assessment via Fine-Grained Domain Adaptation [IJCAI2020]
  • Clarinet: A One-step Approach Towards Budget-friendly Unsupervised Domain Adaptation [IJCAI2020]
  • Weakly-Supervised Domain Adaptation via GAN and Mesh Model for Estimating 3D Hand Poses Interacting Objects [CVPR2020]
  • One-Shot Domain Adaptation for Face Generation [CVPR2020]
  • Learning Meta Face Recognition in Unseen Domains [CVPR2020 Oral] [code]
  • Cross-Domain Document Object Detection: Benchmark Suite and Method [CVPR2020] [code]
  • StereoGAN: Bridging Synthetic-to-Real Domain Gap by Joint Optimization of Domain Translation and Stereo Matching [CVPR2020]
  • Domain Adaptation for Image Dehazing [CVPR2020]
  • Probability Weighted Compact Feature for Domain Adaptive Retrieval [CVPR2020] [code]
  • Disparity-Aware Domain Adaptation in Stereo Image Restoration [CVPR2020]
  • Multi-Path Learning for Object Pose Estimation Across Domains [CVPR2020]
  • Unsupervised Domain Adaptation for 3D Human Pose Estimation [ACM MM2019]
  • PointDAN: A Multi-Scale 3D Domain Adaption Network for Point Cloud Representation [NeurIPS 2019] [code]
  • Deep Head Pose Estimation Using Synthetic Images and Partial Adversarial Domain Adaption for Continuous Label Spaces [ICCV2019]
  • Cross-Domain Adaptation for Animal Pose Estimation [ICCV2019]
  • GA-DAN: Geometry-Aware Domain Adaptation Network for Scene Text Detection and Recognition [ICCV2019]
  • Accelerating Deep Unsupervised Domain Adaptation with Transfer Channel Pruning [IJCNN]
  • Adversarial Adaptation of Scene Graph Models for Understanding Civic Issues [WWW2019]
  • Cross-Dataset Adaptation for Visual Question Answering [CVPR2018]
  • Cross-domain fault diagnosis through optimal transport for a CSTR process [DYCOPS2022] [Code]

Journal

Arxiv

Benchmarks

Library

Lectures and Tutorials

  • A Primer on Domain Adaptation [PDF]

Other Resources

Posted on
Symbols count in article: 35 Reading time ≈ 1 mins.

Bangdream Game scripts Dataset is extracted from Bestdori. It have emotional labels for ZH, JP, EN, KR and TW corpus.
You can obtain wav files for each characters from the voice datas extracted from Game.

Posted on In
Symbols count in article: 139 Reading time ≈ 1 mins.

扩散模型(Diffusion Model)详解：直观理解、数学原理、PyTorch 实现

.

DDIM 简明讲解与 PyTorch 实现：加速扩散模型采样的通用方法

.

Stable Diffusion 解读（一）：回顾早期工作

.

Stable Diffusion 解读（二）：论文精读

.

Stable Diffusion 解读（三）：原版实现及Diffusers实现源码解读

.

LoRA 在 Stable Diffusion 中的三种应用：原理讲解与代码示例

.

Stable Diffusion 中的自注意力替换技术与 Diffusers 实现

.

Stable Diffusion 3 论文及源码概览

.

定制适合自己的 Diffusers 扩散模型训练脚本

.

Posted on In
Symbols count in article: 2.7k Reading time ≈ 10 mins.

2023/2/10更新 vits-onnx 一键式启动

2023/2/17更新 弃用renpy 采用桌宠版本

live2d启动器：克隆Live2DMascot仓库后，修改config.json文件

1
2
3
4
5
6
7
8
9
10
11
"ChatAPI" : 
{
	"ChatSavePath" : "chat",  //聊天音频和文本保存路径
	"CustomChatServer" : 
	{
		"HostPort" : "http://127.0.0.1:8080",  //服务器地址，端口默认8080
		"On" : true,  //开启自定义聊天接口
		"ReadTimeOut" : 114,  //等待响应时间(s)
		"Route" : "/chat"  //路径
	},
}

选择1：server端启动后端api程序(Windows也可以)

Combining chatgpt/gpt3&vits as api and launch it（Server suggested）

将inference_api.py/inference_api_single_speaker.py丢入你的vits项目或moegoe项目中

1
2
3
4
5
6
7
8
9
10
11
12
git clone path/to/your/vits
cd vits
python inference_api.py --model path/to/vits_model.pth --cfg path/to/vits_config.json --key your_openai_api_key
#Single speaker
python inference_api_single_speaker.py --model path/to/vits_model.pth --cfg path/to/vits_config.json --key your_openai_api_key
#修改: 从68行开始，如果该模型用到了类似中日双语cleaner，恢复
#text = f"[JA]{text}[JA]" if is_japanese(text) else f"[ZH]{text}[ZH]"
#选择speaker
speaker_id = 1
#可自行添加chatbot
#此处用到的 
text = gpt3_chat(text)

选择2：绿皮思路chat

支持从外部启动任何正在运行的live2d模型，比如说修改点击事件的对应音频来实现。只需在你的vits项目中加入inference_ork.py这个小文件，然后启动它。注意，需要你能够在自己的windows上部署vits项目，推荐安装好cuda，教程 

1
2
3
4
5
# Nijigasaki COC跑团版
git clone https://huggingface.co/spaces/Mahiruoshi/Lovelive_Nijigasaki_VITS
cd Lovelive_Nijigasaki_VITS
pip install -r requirements.txt
python inference_ork.py

许多live2d都有触碰事件，那么，只需要有一个装载有完整cubism for native功能的live2d播放器,其可以支持点击事件并且支持对口型功能。
此时只需要启动 inference_ork.py
Image text

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
#在程序中将audio参数修改为触碰动作的音频路径
#比如
parser.add_argument('--audio',
                    type=str,
                    help='中途生成的语音存储路径',
                    default = '110Yuki-Setsuna/sounds/temp.wav')
#其在live2d的model3.json文件中对应的是
"TapBody": [
        {
          "File": "motions/a.motion3.json",
          "Sound": "sounds/temp.wav",
          "Text": "ﾋﾄﾘﾀﾞｹﾅﾝﾃｴﾗﾍﾞﾅｲﾖｰ"
        },
        {
          "File": "motions/a.motion3.json",
          "Sound": "sounds/temp2.wav",
          "Text": "ﾋﾄﾘﾀﾞｹﾅﾝﾃｴﾗﾍﾞﾅｲﾖｰ"
        }，
	{
          "File": "motions/a.motion3.json",
          "Sound": "sounds/temp3.wav",
          "Text": "ﾋﾄﾘﾀﾞｹﾅﾝﾃｴﾗﾍﾞﾅｲﾖｰ"
        }
	......]

这样做的本质是让这个绿皮程序不断修改音频文件

1
2
3
4
5
6
7
8
9
10
11
#绿皮启动器的核心
cmd = 'ffmpeg -y -i ' +  args.outdir + '/temp1.wav' + ' -ar 44100 '+ args.audio
#主程序就长这样
def main():
    while True:
      text = input("You:")
      text = infer(text)
      print('Waifu:'+text.replace("[ZH]","").replace("[JA]",""))
    
if __name__ == '__main__':
    main()

如何为gpt-3.5添加设定以达到记忆的功能

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
#比如说我需要和珠绪学姐玩跑团游戏
messages = [
    {"role": "system", "content": "你是超级ai，名字叫巴珠绪。接下来我们将进行一个克苏鲁跑团游戏，你负责扮演守密人，我负责扮演调查员。接下来你会加载一个名叫《幽暗之门》的模组，作为守密人，你需要基于“克苏鲁神话角色扮演游戏规则第七版（Call of Cthulhu 7th Edition）”，我会给你剧本的开头部分，然后基于你对它的理解自由发挥。投掷骰子的环节将由你来模拟，用两个1d10的骰子来生成0-100的随机数。比如说我的某一项属性点是80，当骰子的数目小于80时就判定为成功，0-5为大成功，95-100为大失败等等。整个游戏过程将类似于你来描述故事，我来投骰子并且做出决定来推动剧情的走向。"},
    {'role': 'assistant', 'content': '我明白了，现在我将扮演守密人。'},
    ]
#由于max token的限制，不得不对请求内容进行提纯，但是也能达到记忆的效果。比如这个绿皮程序就会创建一个log.txt来记载设定。
{'role': 'system', 'content': '你是超级ai，名字叫巴珠绪，是我的女朋友。接下来我们将进行一个克苏鲁跑团游戏，你负责扮演守密人，我负责扮演调查员。我会提前告诉你这个故事的真相，作为守密人，你需要基于“克苏鲁神话角色扮演游戏规则第七版（Call of Cthulhu 7th Edition）”，基于你对它的理解，与我一起构建一个新的故事。基于规则，调查员一开始与该事件毫无关联，而你则知道该故事的真相。作为守秘人，你会通过制造一系列事件来向调查员透露事件的线索，来引导调查员完成探索，并且负责新故事的叙述。是战斗、侦察、灵感这种行动的结果通过投掷骰子决定，这个环节将由你来模拟，用两个1d10的骰子来生成0-100的随机数。比如说我的某一项属性点是80，当骰子的数目小于80时就判定为成功，0-5为大成功，95-100为大失败等等。整个游戏过程将就是你来描述故事，引导我做出选择。我做出决定来推动剧情的走向。'}
{'role': 'assistant', 'content': '好的，我需要先了解这个故事的真相'}
{'role': 'system', 'content': '1997年，同为九岁的三个孩子，香川绚郁、织机麻耶、南云涉是玩伴。\n\n三个孩子都来自单亲家庭，不受其他孩子的欢迎。织机性格热烈而大胆，可以说是另外两人的保护者。孩子们在海滩边游玩的时候捡到了漂流而来的鸡蛋模样的梦境晶化器和随之而来的水母守护神。水母守护神希望得到梦境，于是孩子们抱着好玩的心态，用晶化器容纳了织机的梦。\n\n不久之后，织机麻耶单亲母亲不管束的情况下到建筑工地游玩，因高空坠落的钢筋而死。\n\n南云涉发现晶化器之中仍然保存着她的梦境，留恋着织机麻耶死亡了的梦。即使在逐渐\n\n长大后，也无法走出过去面向未来。与此相反，不知情的香川对过去的同伴抱着不同的态度。\n\n水母守护神则消极怠工地守护着这片梦境。\n\n寄住在别人家中的南云涉无处存放晶化器，升入高中后他将晶化器藏匿在高中的躲猫猫社团内。\n\n时间来到2005年，偶然注意到下关市的奈亚拉托提普发觉了从幻梦境出逃的水母守护神以及被人类使用过的晶化器，奈亚在经过羽间高中学校门口时候，随意使用了一名离开学校的学生的样貌（宇白和也），进入高中后对幻梦境存有恶意的奈亚随手撕裂了水母守护神，恶趣味的无貌之神杀死了南云涉，把他的灵魂塞入了以水母尸体构建的死者梦境，丢弃了水母守护神的尸体，取走了晶化器，奈亚化为的宇白和也被真正宇白和也的朋友辻本茜撞见，便也随手杀死了她，同时用水母的尸体拼凑了茜。\n\n实质上已经死去的茜因此在几日之内无神地夜游着，仅仅是成为拼凑物的她却得到了不可思议的能力，自称占卜的魔女。\n\n而目睹了南云涉尸体的川澄美沙（报案人），被校方委托，找寻几名学生参加南云涉的追悼会，这也是调查员们涉入的开始。'}
{'role': 'assistant', 'content': '明白了，接下来我会假装你对此事一无所知，开始扮演调查员的角色。而我将扮演守秘人，开始这次跑团。我会使用克苏鲁风格的描述来营造恐怖感，因此来基于这次游戏更好的体验。接下来请创建你的调查员'}
{'role': 'user', 'content': '我叫大场奈奈，是一个女高中生。所有属性都是80，只要投掷骰子的结果小于80，就判断为成功。'}
{'role': 'assistant', 'content': '好的，大场奈奈，你是一个在学校里相当受欢迎的女高中生。你和你的几个学生朋友被学校委托参加南云涉的追悼会。在会上，你听到了一些让你感到不安的事情。你发现南云涉的死亡似乎不是一起普通的意外事故，你决定开始展开调查。现在你身处于学校的教学楼，准备开始你的调查。你要采取什么行动呢？'}
{'role': 'user', 'content': '我想起了之前从舞台创造科拿到的页锤，这是一个很不错工具，可以破坏掉障碍物'}
{'role': 'assistant', 'content': '你拿出了页锤，准备用它破开通风管道的洞口。你用力敲打，洞口终于被打开了。你和你的朋友钻进了通风管道，小心翼翼地穿过长长的通道，直到你们看到了实验室的入口。\n\n实验室内非常昏暗，你们看到了许多发着微光的管路和仪器设备，发现实验室内存在大量类似于南云涉在房间里发现的尸体和怪异生物的存在。\n\n你们在实验室中进行搜索，发现了一张日记本，记录了实验室的过去和实验者们进行的一些实验。然而，日记中的部分内容变得越来越奇怪，好像涉及到一些神秘的力量存在。\n\n你和你的朋友突然听到了一个诡异的声响，似乎是一个陌生的人在接近你们。你们不知道是什么，但是肯定不想被它发现。你决定采取什么行动来躲避这个陌生的人？'}
{'role': 'user', 'content': '我要过尝试能不能隐藏起来，我的隐藏点数有80点，应该可以成功'}
{'role': 'assistant', 'content': '你和你的朋友匆忙寻找躲藏的地方。你们找到了一扇半开着的储藏室，里面有很多杂物，你们钻了进去，静静地等待着。你们听到了脚步声和沙沙声，似乎那个陌生的人正在进行搜查。\n\n幸运的是，你和你的朋友成功地躲了过去，那个人没有发现你们。不久后，他走了出去，你们继续进行调查。\n\n你们找到了很多关于南云涉死亡的证据，但是这些证据与日记中提到的神秘力量之间似乎存在一些联系。你们决定继续深入调查，寻找更多的线索。'}
#重新启动时，用它替换原先的messages，就可以断点重连了，记得自行修改格式

对于 text_to_sequence相关错误

1
2
3
4
5
6
7
#在推理中，可能出现symbols相关错误，这主要是由于不同text cleaner之间的冲突导致的
line 85, in infer
seq = text_to_sequence
#需要你在这一行自行修改
#如果需要
symbols seq = text_to_sequence(text, symbols=hps.symbols, cleaner_names=hps.data.text_cleaners)
#如不需要，把 symbols=hps.symbols 删掉

Posted on In
Symbols count in article: 957 Reading time ≈ 3 mins.

邦邦特化AI聊天(2代live2d)

本分支为适应本地快速部署的轻量化版本，可从main分支或者其它分支查看中小型网页部署版或开发版 

1
2
3
4
#本机部署
git clone https://github.com/Paraworks/BangDreamAi
pip install -r requirements.txt
python app.py

默认5000端口

配置live2d模型

BangDream的live2d可以直接从bestdori手动下载。Github上也有许多提取好的仓库

五团(卡面比较多)

七团

b站北极暖水港大佬

你需要将这些模型放在/static/Resources下

按照如下标准

1
/Resources/{乐队}/{角色}/{模型名}/model.json

这样程序可以自动识别乐队和角色,否则无法在前端选择

模型配置

如果没什么大问题，可以直接在前端页面配置，但也可以本地直接修改config.json

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
默认配置
{
    "api": "chatgpt",
    "apiKey": "",
    "apiLink": "",
    "band": "Afterglow",
    "duration": 2,
    "frequence": 0.5,
    "modelPath": "static/Resources/Afterglow/蘭/live/model.json",
    "mouseTrack": 1,
    "positionX": 300,
    "positionY": 50,
    "scale": 0.3,
    "speaker": "蘭",
    "stopBreath": 0,
    "text": {
        "response": "初始化成功",
        "expression": null,
        "motion": null
    },
    "textApiBaseUrl": "http://127.0.0.1:5000/api/sentence/test",
    "ttsApiBaseUrl": "http://127.0.0.1:8000/?is_chat=false",
    "volum": 70
}

启动TTS合成

TTS(Bang dream Bert vits示例):

1
2
3
4
5
6
git clone https://huggingface.co/spaces/Mahiruoshi/MyGO_VIts-bert
cd MyGO_VIts-bert
pip install -r requirements.txt
pip install Flask
pip install Flask-CORS
python server.py

自定义chatbot

由于在前端配置完apikey后，直接跑chatgpt也大概率跑不通,建议自行修改chat.py的相关配置以及逻辑

To do list

自定义直播间管人配置(未完成)

1
{url}/{session_id}/{task_id}

OBS 配置详解：

live2d 配置

  • 来源 > + > 浏览器 > 新建 > URL：填入上一步生成的live2d链接，一般是localhost:xxxx

弹幕姬

  • 来源 > + > 浏览器 > 新建 > URL：参照说明填入链接以及css

B 站推流配置

  • (obs内)设置（Preferences）> 直播 > 服务 > 自定义 > 填写 bilibili服务器以及推流码…
  • 推流码填写「B 站首页 > 头像 > 推荐服务 > 直播中心 > 左侧“我的直播间”> 填好直播分类、房间标题 > 开始直播，然后会显示的串流密钥」

开发历程

服务器部署中踩过的一些坑:
TTS模型非常吃内存，而中文的bert模型大小是日语的几倍，在我使用的4核4G内存的服务器上就只能进行日语TTS了。
就目前而言推荐用Mygo的bert模型，不论是大小还是效果都理想一些。

技术细节:用pixi-live2d-display驱动live2d

live2d我之前也没碰过，就跟着这个项目走了下来，然后发现这玩意整成皮套人可能还是最容易的。然后我还是遇到了那个问题。
pixi-live2d-display不支持口型同步，其实cubism本体也需要自己手搓相关代码。
于是我找到了这个issue
每个live2d模型都有对张嘴角度的设定，也就是lip

1
2
model.internalModel.coreModel.setParameterValueById('ParamMouthOpenY',v);

而bang dream使用的是二代模型，参数不通用

1
model.internalModel.coreModel.setParamFloat('PARAM_MOUTH_OPEN_Y', v);

之后要做的事说白了就是先调用函数分析wav文件的响度，将其转化成lip参数对应的区间(0~1)

1
2
3
4
5
6
7
8
9
10
11
const run = () => {
    if (!playing) return;
    const frequencyData = getByteFrequencyData(analyser, new Uint8Array(analyser.frequencyBinCount));
    const arr = [];
    for (let i = 0; i < 1000; i += o) {
        arr.push(frequencyData[i]);
    }
    setMouthOpenY((arrayAdd(arr) / arr.length - 20) / 60);
    setTimeout(run, 1000 / 30);
};
run();

因为所用到的是wav文件，需要对参数进行微调。
这样就可以让皮套人说话了，就是index.js
程序会向目标端口发送请求获取待合成文本

1
2
3
4
5
6
7
8
9

let text;
try {//从launcher.py启动的flask服务中获取待读文本
    const response = await fetch('http://127.0.0.1:5180/show');
    text = await response.text();
} catch (error) {
    console.error('Failed to get text from the server', error);
    return;
}

之后将其发送给TTS端

1
request.open('GET', `http://127.0.0.1:5000/tts?text=${encodeURIComponent(text)}&speaker=ましろ`, true);

实现

综上:
前端组件+chatot程序+live2d+TTS = 服务器或本地端”AI vtuber”
弹幕监听+chatbot(两者可以直接整合在一起)+TTS+vtuber = AI Vtuber

Posted on In
Symbols count in article: 1.4k Reading time ≈ 5 mins.

Bang Dream全员TTS

huggingface

使用须知

由于bert-vits项目才刚刚支持日语，所以稳定度和效果都会有影响。
参数设置:可以视情况将感情调节和音素控制调节至0.3以下，对应非日常语言(小说等)拥有更好的效果。

使用Galgame引擎进行二创

WebGAL

将AI皮套人部署到服务器或者直播间

轻量化live2d驱动

长文本自动合成：

选择1:自定义文本

与训练用的标记方法相似，用”|”标识符将说话人和内容分割开,参考圣经.txt

1
2
3
4
5
6
素世|我也会好好跟你说明我不得不组乐队的理由
素世|我想如果你能见我一面，你就一定能明白的
素世|我是小祥你的同伴
素世|我好想见你
祥子|真是会虚情假意呢
祥子|想演奏是你们的自由，你们就请便吧

实际上并不需要在意”祥子”或者”素世“这些说话人是什么，名字仅仅是用来标记的。
比如

1
2
3
4
5
6
为|我也会好好跟你说明我不得不组乐队的理由
什|我想如果你能见我一面，你就一定能明白的
么|我是小祥你的同伴
要|我好想见你
演奏|真是会虚情假意呢
春日影|想演奏是你们的自由，你们就请便吧

重要的是在 拓展功能>>>>>>角色对应表中进行手动设置
详见config下的”spk2id”项。对于算法来说，你实际上是在完成一个
“说话人所扮演的角色名”—>”说话人”—>”编号”
作为导演，需要让说话人扮演你所希望的角色：

1
2
3
4
5
6
7
8
そよ|素世
祥子|祥子
ましろ|为
七深|什
透子|么
つくし|要
瑠唯|演奏
そよ|春日影

选择2:游戏脚本自动解读

其实我不知道让她们重新读一遍日语有什么用，但因为不用做任何映射关系，直接去下载一个日语脚本丢进去就好了。
当然，可以让她们读中文脚本，帮助入眠的同时又过了遍剧情()。
下载台服脚本
设置角色对应关系(其实就是译名):

1
2
3
4
5
ましろ|真白
七深|七深
透子|透子
つくし|筑紫
瑠唯|瑠唯

选择3:读小说(建议本地)

支持epub、txt和mobi，但出于稳定性考虑建议先自行用工具转换成txt格式。
当然，在网页端是无法完整合成的，确保在本地运行并且安装了gpu支持(也就是cuda和cudnn)。
所有音频都会被存储在”BangDream-Bert-VITS2/books”文件夹下，记得合成后转移已生成文件，再次合成时将会自动清空。

本地运行:

1
2
3
4
git clone https://huggingface.co/spaces/Mahiruoshi/BangDream-Bert-VITS2
cd BangDream-Bert-VITS2
pip install -r requirements.txt
python app.py

样例API 使用说明

Flask启动代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
logger = logging.getLogger(__name__)
import datetime
import numpy as np
import torch
from ebooklib import epub
import PyPDF2
from PyPDF2 import PdfReader
import zipfile
import shutil
import sys, os
import json
from bs4 import BeautifulSoup
import argparse
import commons
import utils
from models import SynthesizerTrn
from text.symbols import symbols
from text import cleaned_text_to_sequence, get_bert
from text.cleaner import clean_text
import gradio as gr
import webbrowser
import re
from scipy.io.wavfile import write

net_g = None

if sys.platform == "darwin" and torch.backends.mps.is_available():
    device = "mps"
    os.environ["PYTORCH_ENABLE_MPS_FALLBACK"] = "1"
else:
    device = "cuda"

def is_japanese(string):
        for ch in string:
            if ord(ch) > 0x3040 and ord(ch) < 0x30FF:
                return True
        return False

def get_text(text, language_str, hps):
    norm_text, phone, tone, word2ph = clean_text(text, language_str)
    phone, tone, language = cleaned_text_to_sequence(phone, tone, language_str)

    if hps.data.add_blank:
        phone = commons.intersperse(phone, 0)
        tone = commons.intersperse(tone, 0)
        language = commons.intersperse(language, 0)
        for i in range(len(word2ph)):
            word2ph[i] = word2ph[i] * 2
        word2ph[0] += 1
    bert = get_bert(norm_text, word2ph, language_str, device)
    del word2ph
    assert bert.shape[-1] == len(phone), phone

    if language_str == "ZH":
        bert = bert
        ja_bert = torch.zeros(768, len(phone))
    elif language_str == "JA":
        ja_bert = bert
        bert = torch.zeros(1024, len(phone))
    else:
        bert = torch.zeros(1024, len(phone))
        ja_bert = torch.zeros(768, len(phone))

    assert bert.shape[-1] == len(
        phone
    ), f"Bert seq len {bert.shape[-1]} != {len(phone)}"

    phone = torch.LongTensor(phone)
    tone = torch.LongTensor(tone)
    language = torch.LongTensor(language)
    return bert, ja_bert, phone, tone, language


def infer(text, sdp_ratio, noise_scale, noise_scale_w, length_scale, sid, language):
    global net_g
    bert, ja_bert, phones, tones, lang_ids = get_text(text, language, hps)
    with torch.no_grad():
        x_tst = phones.to(device).unsqueeze(0)
        tones = tones.to(device).unsqueeze(0)
        lang_ids = lang_ids.to(device).unsqueeze(0)
        bert = bert.to(device).unsqueeze(0)
        ja_bert = ja_bert.to(device).unsqueeze(0)
        x_tst_lengths = torch.LongTensor([phones.size(0)]).to(device)
        del phones
        speakers = torch.LongTensor([hps.data.spk2id[sid]]).to(device)
        audio = (
            net_g.infer(
                x_tst,
                x_tst_lengths,
                speakers,
                tones,
                lang_ids,
                bert,
                ja_bert,
                sdp_ratio=sdp_ratio,
                noise_scale=noise_scale,
                noise_scale_w=noise_scale_w,
                length_scale=length_scale,
            )[0][0, 0]
            .data.cpu()
            .float()
            .numpy()
        )
        del x_tst, tones, lang_ids, bert, x_tst_lengths, speakers
        return audio


def tts_fn(
    text, speaker, sdp_ratio, noise_scale, noise_scale_w, length_scale, language
):
    with torch.no_grad():
        audio = infer(
            text,
            sdp_ratio=sdp_ratio,
            noise_scale=noise_scale,
            noise_scale_w=noise_scale_w,
            length_scale=length_scale,
            sid=speaker,
            language= "JP" if is_japanese(text) else "ZH",
        )
        torch.cuda.empty_cache()
        write("books/temp.wav", 44100, audio)
    return "Success"

app = Flask(__name__)

@app.route('/tts')
def tts_api():
    # 从请求中获取参数
    text = request.args.get('text')
    speaker = request.args.get('speaker')
    sdp_ratio = float(request.args.get('sdp_ratio', 0.2))
    noise_scale = float(request.args.get('noise_scale', 0.6))
    noise_scale_w = float(request.args.get('noise_scale_w', 0.8))
    length_scale = float(request.args.get('length_scale', 1))
    
    status = tts_fn(text, speaker='つくし', sdp_ratio=0.2, noise_scale=0.6, noise_scale_w=0.8, length_scale=1)
    with open('books/temp.wav','rb') as bit:
        wav_bytes = bit.read()
    
    headers = {
            'Content-Type': 'audio/wav',
            'Text': status.encode('utf-8')}
    return wav_bytes, 200, headers

if __name__ == "__main__":
    parser = argparse.ArgumentParser()
    parser.add_argument(
        "-m", "--model", default="./logs/BangDream/G_47000.pth", help="path of your model"
    )
    parser.add_argument(
        "-c",
        "--config",
        default="./logs/BangDream/config.json",
        help="path of your config file",
    )
    parser.add_argument(
        "--share", default=True, help="make link public", action="store_true"
    )
    parser.add_argument(
        "-d", "--debug", action="store_true", help="enable DEBUG-LEVEL log"
    )

    args = parser.parse_args()
    if args.debug:
        logger.info("Enable DEBUG-LEVEL log")
        logging.basicConfig(level=logging.DEBUG)
    hps = utils.get_hparams_from_file(args.config)

    device = (
        "cuda:0"
        if torch.cuda.is_available()
        else (
            "mps"
            if sys.platform == "darwin" and torch.backends.mps.is_available()
            else "cpu"
        )
    )
    net_g = SynthesizerTrn(
        len(symbols),
        hps.data.filter_length // 2 + 1,
        hps.train.segment_size // hps.data.hop_length,
        n_speakers=hps.data.n_speakers,
        **hps.model,
    ).to(device)
    _ = net_g.eval()

    _ = utils.load_checkpoint(args.model, net_g, None, skip_optimizer=True)

    app.run(host="0.0.0.0", port=8080)

API 请求路径：

1
/tts

请求方法：

1
GET

请求参数:

1
2
3
4
5
6
text (必须) - 您想要转化为语音的文本。
speaker (可选) - 选择的发音人，默认为 ‘つくし’。
sdp_ratio (可选) - 参数值，默认为 0.2。
noise_scale (可选) - 噪音比例，默认为 0.6。
noise_scale_w (可选) - 噪音比例宽度，默认为 0.8。
length_scale (可选) - 长度比例，默认为 1。

响应：

1
响应为 WAV 格式的音频文件，可以直接播放。

响应头：

1
2
Content-Type: ‘audio/wav’ - 表示响应内容为 WAV 格式的音频文件。
Text: 转化的状态，编码为 UTF-8。

调用示例：

1
GET /tts?text=你好，世界！&speaker=つくし&sdp_ratio=0.3&noise_scale=0.5

Posted on In
Symbols count in article: 2.3k Reading time ≈ 8 mins.

爬虫结果
直接拿去炼TTS或svc啥的都可以

前期调查
由于没有mygo看，我决定炼一个mygo全员的tts模型来给自己造糖吃。回想起之前炼虹团时夙兴夜寐地解析手游资源的痛苦时光。与少歌一样，bestdori平台可以省下解包手游的精力。
游戏脚本中包含了制作数据集所需的所有信息。
而存储音频文件的路径也能在平台上找到。
我已经快等不及了，快点爬下来把数据集端上来罢。
然后我品鉴了一下这令人忍俊不禁的命名方式:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
-scenario
    area_opening_story34
    area_opening_story35
    area_opening_story7
    area_opening_story9
    backstagestory1
    bandstory1
    bandstory10
    bandstory100
    bandstory101
    bandstory102
    bandstory103
    bandstory104
    ......
    eventstory199_3
    eventstory199_4
    eventstory199_5
    eventstory199_6
    eventstory2_0
    eventstory2_1
    eventstory2_2
    eventstory2_3
    eventstory2_4
    eventstory2_5
    ......

狠狠地切割了。
看来是没法通过总结命名方法整到所有音频文件夹的详细地址。那么还是准备暴力破解吧。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
# 分析
要用到的python包主要有beautifulsoup4、requests还有selenium。
先试着直接解析。

import requests
from bs4 import BeautifulSoup

# 网址
URL = "https://bestdori.com/tool/explorer/asset/jp/sound/voice/scenario/bandstory294"

# 使用 requests 库来获取网页内容
response = requests.get(URL)
response.raise_for_status()

# 解析 HTML
soup = BeautifulSoup(response.content, 'html.parser')
print(soup.prettify())

先让我们看看解析的结果。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
<body>
  <noscript>
   <strong>
    We're sorry but Bestdori! doesn't work properly without JavaScript enabled. Please enable it to continue.
   </strong>
  </noscript>
  <div id="app">
  </div>
  <script src="/js/chunk-vendors.9309a223.js">
  </script>
  <script src="/js/app.3b5d5ef2.js">
  </script>
  <script crossorigin="anonymous" data-cf-beacon='{"rayId":"80e4bf36cdaf49cc","version":"2023.8.0","r":1,"b":1,"token":"6e9e54182ea54522b7f2b9e65e87b87e","si":100}' defer="" integrity="sha512-bjgnUKX4azu3dLTVtie9u6TKqgx29RBwfj3QXYt5EKfWM/9hPSAI/4qcV5NACjwAo8UtTeWefx6Zq5PHcMm7Tg==" src="https://static.cloudflareinsights.com/beacon.min.js/v8b253dfea2ab4077af8c6f58422dfbfd1689876627854">
  </script>
 </body>

解析了吗？如解，除了body part外都有了。很明显，这并不是我想要的结果。

网站爬取

为了解析动态网站，需要先下载googledriver,同时用pip install安装先前提到的selenium包。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
from selenium import webdriver
from bs4 import BeautifulSoup
#测试用的网址
URL = "https://bestdori.com/tool/explorer/asset/jp/sound/voice/scenario/bandstory294"

# 指定chromedriver的路径
driver_path = r'C:\Users\Admin\爬虫\chromedriver-win64\chromedriver.exe'
driver = webdriver.Chrome(executable_path=driver_path)

# 打开指定的URL
driver.get(URL)

# 等待页面完全加载，这里可以加上适当的延时，或者使用Selenium的等待机制
# 简单的延时示例（等待10秒）：
import time
time.sleep(10)

# 获取页面的HTML内容
page_source = driver.page_source

# 使用BeautifulSoup解析
soup = BeautifulSoup(page_source, 'html.parser')
print(soup.prettify())

# 关闭浏览器
driver.quit()

这次的结果就和谷歌浏览器右键检查的结果一样了，直接看我们想要的。

1
2
3
4
5
6
7
8
9
10
<div class="m-b-l">
          <div class="m-b-xs">
           <span class="icon">
            <i class="fas fa-file-audio">
            </i>
           </span>
           <span>
            event57-07-048.mp3
           </span>
          </div>

“icon”和”fas fa-file-audio”就是用来检索的标签。

1
2
3
4
5
6
7
8
9
10
11
12
from bs4 import BeautifulSoup

html_content = soup.prettify()

soup = BeautifulSoup(html_content, 'html.parser')

# 查找所有具有fas fa-file-audio图标的<span>标签的下一个兄弟标签
spans = soup.find_all('span', class_='icon')
file_names = [span.find_next_sibling().text.strip() for span in spans if span.find('i', class_='fas fa-file-audio')]

for file_name in file_names:
    print(file_name)

看一下检索的结果

1
2
3
4
5
6
event57-07-001.mp3
event57-07-002.mp3
......
event57-07-048.mp3
event57-07-049.mp3
......

这就是我想要的
同理在上级目录下把”fas fa-file-audio”换成”fas fa-file-archive”

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
from selenium import webdriver
from bs4 import BeautifulSoup

URL = "https://bestdori.com/tool/explorer/asset/jp/sound/voice/scenario"

# 指定chromedriver的路径
driver_path = r'C:\Users\Admin\爬虫\chromedriver-win64\chromedriver.exe'
driver = webdriver.Chrome(executable_path=driver_path)

# 打开指定的URL
driver.get(URL)

# 等待页面完全加载，这里可以加上适当的延时，或者使用Selenium的等待机制
# 简单的延时示例（等待10秒）：
import time
time.sleep(10)

# 获取页面的HTML内容
page_source = driver.page_source

# 使用BeautifulSoup解析
soup = BeautifulSoup(page_source, 'html.parser')

# 关闭浏览器
driver.quit()

html_content = soup.prettify()

soup = BeautifulSoup(html_content, 'html.parser')

# 查找所有具有fas fa-file-archive图标的<span>标签的下一个兄弟标签
spans = soup.find_all('span', class_='icon')
file_names = [span.find_next_sibling().text.strip() for span in spans if span.find('i', class_='fas fa-file-archive')]

for file_name in file_names:
    print(file_name)

就能提取上级目录的名字了。

完整代码

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
from selenium import webdriver
from bs4 import BeautifulSoup
import time

# 设置chromedriver的路径
driver_path = r'C:\Users\Admin\爬虫\chromedriver-win64\chromedriver.exe'
driver = webdriver.Chrome(executable_path=driver_path)

# 1. 访问主页面并获取所有子目录的名称
URL = "https://bestdori.com/tool/explorer/asset/jp/sound/voice/scenario"
driver.get(URL)
time.sleep(2)  # 等待页面加载
soup = BeautifulSoup(driver.page_source, 'html.parser')
subdirs = [span.find_next_sibling().text.strip() for span in soup.find_all('span', class_='icon') if span.find('i', class_='fas fa-file-archive')]

# 为保存mp3路径的txt文件做准备
with open("C:/Users/Admin/爬虫/WholeMp3UrlPaths.txt", "w") as f:

    # 2. 对于每个子目录，获取所有mp3文件的名称
    for subdir in subdirs:
        URL_subdir = f"https://bestdori.com/tool/explorer/asset/jp/sound/voice/scenario/{subdir}"
        driver.get(URL_subdir)
        time.sleep(2)  # 等待页面加载
        soup = BeautifulSoup(driver.page_source, 'html.parser')
        mp3_names = [span.find_next_sibling().text.strip() for span in soup.find_all('span', class_='icon') if span.find('i', class_='fas fa-file-audio')]

        # 3. 保存mp3文件的路径到txt文件
        for mp3_name in mp3_names:
            f.write(f"https://bestdori.com/assets/jp/sound/voice/scenario/{subdir}_rip/{mp3_name}\n")

# 关闭浏览器
driver.quit()

所有语音的路径就被记录在WholeMp3UrlPaths.txt里面了，需要注意的是，真正获取音频文件的路径并非简单的加法，而是

1
f.write(f"https://bestdori.com/assets/jp/sound/voice/scenario/{subdir}_rip/{mp3_name}\n")

解析动态网址的耗时非常漫长，而且此时的任务只完成了一半，还需要对asset内的信息进行整理。

下载所有剧情脚本

游戏内的剧情脚本类似于galgame(实际上复杂多了，主要还有live2d动作之类的)。
用与获取mp3音频路径同样的方式下载所有脚本。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
from selenium import webdriver
from bs4 import BeautifulSoup
import time
import requests
import os

# 设置chromedriver的路径和初始URL
driver_path = r'C:\Users\Admin\爬虫\chromedriver-win64\chromedriver.exe'
base_url = "https://bestdori.com/tool/explorer/asset/jp/scenario"

driver = webdriver.Chrome(executable_path=driver_path)

# 第一层目录
driver.get(base_url)
time.sleep(10)
soup = BeautifulSoup(driver.page_source, 'html.parser')
first_level_dirs = [span.find_next_sibling('span', class_='m-l-xs').text.strip() for span in soup.find_all('span', class_='icon') if (span.find('i', class_='fas fa-file-archive') or span.find('i', class_='fas fa-folder'))]

# 保存所有找到的.asset文件的链接
all_asset_links = []

for first_dir in first_level_dirs:
    driver.get(f"{base_url}/{first_dir}")
    time.sleep(10)
    soup = BeautifulSoup(driver.page_source, 'html.parser')
    second_level_dirs = [span.find_next_sibling('span', class_='m-l-xs').text.strip() for span in soup.find_all('span', class_='icon') if (span.find('i', class_='fas fa-file-archive') or span.find('i', class_='fas fa-folder'))]

    for second_dir in second_level_dirs:
        driver.get(f"{base_url}/{first_dir}/{second_dir}")
        time.sleep(10)
        soup = BeautifulSoup(driver.page_source, 'html.parser')
        links = soup.find_all('a', attrs={'download': True})
        asset_links = ['https://bestdori.com' + link['href'] for link in links]
        print(asset_links)
        all_asset_links.extend(asset_links)

# 下载所有找到的.asset文件
save_path = r'C:\Users\Admin\爬虫\assert'
if not os.path.exists(save_path):
    os.makedirs(save_path)

for link in all_asset_links:
    response = requests.get(link, stream=True)
    filename = os.path.join(save_path, link.split('/')[-1])

    if response.status_code == 200:
        with open(filename, 'wb') as f:
            for chunk in response.iter_content(1024):
                f.write(chunk)

# 关闭浏览器
driver.quit()

标记数据集制作

对剧情脚本重新进行整理，就能得到标记好的数据集了。

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
import os
import json
import re

def extract_data(data, current_data=None, output_file=None):
    if current_data is None:
        current_data = {}

    if isinstance(data, dict):
        if 'windowDisplayName' in data:
            current_data['windowDisplayName'] = data['windowDisplayName']
        if 'body' in data:
            # 移除body中的换行符
            current_data['body'] = data['body'].replace('\n', '')
        if 'voiceId' in data:
            current_data['voiceId'] = data['voiceId']
            
            # 检查所有字段是否非空
            valid_data = all(current_data.get(k) for k in ['windowDisplayName', 'body', 'voiceId'])
            # 检查 windowDisplayName 是否包含 "・"
            valid_displayname = "・" not in current_data.get('windowDisplayName', "")
            # 检查 body 是否只包含标点符号
            valid_body = bool(re.sub(r'[^\w]', '', current_data.get('body', "")))
            
            # 如果满足所有条件，输出结果到文件
            if valid_data and valid_displayname and valid_body:
                output_file.write(f"{current_data['voiceId']}|{current_data['windowDisplayName']}|{current_data['body']}\n")
                current_data.clear()  # 清空当前数据以供下次使用
        
        for key in data:
            extract_data(data[key], current_data, output_file)
    elif isinstance(data, list):
        for item in data:
            extract_data(item, current_data, output_file)

# 获取目录下的所有文件
directory = "C:/Users/Admin/爬虫/assert"
files = os.listdir(directory)

# 打开一个txt文件以保存结果
with open("BangDreamSortPath.txt", "w", encoding="utf-8") as output_file:
    # 遍历所有文件
    for filename in files:
        if filename.endswith(".asset"):
            file_path = os.path.join(directory, filename)
            with open(file_path, 'r', encoding='utf-8') as file:
                content = json.load(file)
                extract_data(content, output_file=output_file)

制作预处理清单
为了方便在colab平台上快速处理数据集，将mp3地址和整理完的”BangDreamSortPath.txt”重新整合:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
# 从 WholeMp3UrlPaths.txt 创建一个映射
path_mapping = {}
with open("WholeMp3UrlPaths.txt", "r", encoding="utf-8", errors="ignore") as f:
    for line in f:
        try:
            audio_id = line.strip().split("/")[-1].replace(".mp3", "")
            path_mapping[audio_id] = line.strip('\n')
        except Exception as e:
            print(f"Error processing line {line}: {e}")

# 遍历 BangDreamSortPath.txt 的每一行，并根据需要替换音频名
new_results = []
with open("BangDreamSortPath.txt", "r", encoding="utf-8") as f:
    for line in f:
        try:
            audio_id = line.split("|")[0]
            if audio_id in path_mapping:
                line = line.replace(audio_id, path_mapping[audio_id])
            new_results.append(line)
        except Exception as e:
            print(f"Error processing line {line}: {e}")

# 将新的结果保存到新的 txt 文件中
with open("SortPathUrl.txt", "w", encoding="utf-8") as f:
    f.writelines(new_results)

# 简单地筛掉无效网址
with open(file_path, 'r', encoding='utf-8') as file:
    lines = file.readlines()

# 只保留以https开头的行
filtered_lines = [line for line in lines if line.startswith('https')]

# 将筛选后的内容写回文件
with open(file_path, 'w', encoding='utf-8') as file:
    file.writelines(filtered_lines)

print("File has been processed.")

这样就能省区繁杂的上传流程而能够直接在colab上进行处理了。

模型训练

网上Bert-VITS2的教程十分脑残，大概是那边不愿意把一键流程流出导致滥用。
先说一下要点。
数据的预处理没上面好说的，用request模块从SortPathUrl.txt下载音频文件时时直接处理成44100的单声道就好了，用不上resample.py
训练前先用preprocess_text.py处理SortPathUrl.txt(记得把里面的网址替换为真实路径)，会自动生成config。
下载日语bert模型，运行bert_gen.py 然后直接训练。
然后这样在train_ms.py def run() 函数的上面加入：

1
2
3
4
5
6
import os

os.environ['MASTER_ADDR'] = '127.0.0.1'
os.environ['MASTER_PORT'] = '8880'
os.environ['WORLD_SIZE'] = '1'
os.environ['RANK'] = '0'

事后记:逆天colab重采样的音频大概率没法保存成功，浪费算力是板上钉钉的了(悲)