用户
 找回密码
 立即注册
发表于 2021-4-8 14:38:15
129926
在使用nemo docker进行训练时,出现以下错误,已确认数据配置正确,按照官方代码训练,请问一下错误出现原因是什么
  1. Validation sanity check: 0it [00:00, ?it/s]Traceback (most recent call last):
  2.   File "train.py", line 26, in <module>
  3.     trainer.fit(quartznet)#调用‘fit’方法开始训练
  4.   File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 445, in fit
  5.     results = self.accelerator_backend.train()
  6.   File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/accelerators/gpu_accelerator.py", line 64, in train
  7.     results = self.train_or_test()
  8.   File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/accelerators/accelerator.py", line 66, in train_or_test
  9.     results = self.trainer.train()
  10.   File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 467, in train
  11.     self.run_sanity_check(self.get_model())
  12.   File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 659, in run_sanity_check
  13.     _, eval_results = self.run_evaluation(test_mode=False, max_batches=self.num_sanity_val_batches)
  14.   File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/trainer.py", line 579, in run_evaluation
  15.     output = self.evaluation_loop.evaluation_step(test_mode, batch, batch_idx, dataloader_idx)
  16.   File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/trainer/evaluation_loop.py", line 171, in evaluation_step
  17.     output = self.trainer.accelerator_backend.validation_step(args)
  18.   File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/accelerators/gpu_accelerator.py", line 88, in validation_step
  19.     output = self.__validation_step(args)
  20.   File "/opt/conda/lib/python3.6/site-packages/pytorch_lightning/accelerators/gpu_accelerator.py", line 96, in __validation_step
  21.     output = self.trainer.model.validation_step(*args)
  22.   File "/opt/conda/lib/python3.6/site-packages/nemo/collections/asr/models/ctc_models.py", line 442, in validation_step
  23.     log_probs, encoded_len, predictions = self.forward(input_signal=signal, input_signal_length=signal_len)
  24.   File "/opt/conda/lib/python3.6/site-packages/nemo/core/classes/common.py", line 511, in __call__
  25.     outputs = wrapped(*args, **kwargs)
  26.   File "/opt/conda/lib/python3.6/site-packages/nemo/collections/asr/models/ctc_models.py", line 396, in forward
  27.     input_signal=input_signal, length=input_signal_length,
  28.   File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 726, in _call_impl
  29.     result = self.forward(*input, **kwargs)
  30.   File "/opt/conda/lib/python3.6/site-packages/nemo/core/classes/common.py", line 511, in __call__
  31.     outputs = wrapped(*args, **kwargs)
  32.   File "/opt/conda/lib/python3.6/site-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
  33.     return func(*args, **kwargs)
  34.   File "/opt/conda/lib/python3.6/site-packages/nemo/collections/asr/modules/audio_preprocessing.py", line 79, in forward
  35.     processed_signal, processed_length = self.get_features(input_signal, length)
  36.   File "/opt/conda/lib/python3.6/site-packages/nemo/collections/asr/modules/audio_preprocessing.py", line 249, in get_features
  37.     return self.featurizer(input_signal, length)
  38.   File "/opt/conda/lib/python3.6/site-packages/torch/nn/modules/module.py", line 726, in _call_impl
  39.     result = self.forward(*input, **kwargs)
  40.   File "/opt/conda/lib/python3.6/site-packages/torch/autograd/grad_mode.py", line 15, in decorate_context
  41.     return func(*args, **kwargs)
  42.   File "/opt/conda/lib/python3.6/site-packages/nemo/collections/asr/parts/features.py", line 375, in forward
  43.     x = normalize_batch(x, seq_len, normalize_type=self.normalize)
  44.   File "/opt/conda/lib/python3.6/site-packages/nemo/collections/asr/parts/features.py", line 60, in normalize_batch
  45.     "normalize_batch with `per_feature` normalize_type received a tensor of length 1. This will result "
  46. ValueError: normalize_batch with `per_feature` normalize_type received a tensor of length 1. This will result in torch.std() returning nan
复制代码




使用道具 举报 回复
发表于 2021-4-8 17:54:09
本帖最后由 zhiyuan 于 2021-4-8 18:02 编辑

网络上搜不到这个错误,不知道是哪里出了问题,附上代码

nemo_hackathon_final_3.28.ipynb.zip

449.36 KB, 下载次数: 4

使用道具 举报 回复 支持 反对
发表于 2021-4-8 17:58:48
能否提供完整的执行代码过程,执行哪一步出现的报错?
使用道具 举报 回复 支持 反对
发表于 2021-4-8 20:09:13
zhiyuan 发表于 2021-4-8 17:54
网络上搜不到这个错误,不知道是哪里出了问题,附上代码

试了你的代码 , 问题出现在你的数据清单,那个train.json 和test.json的文件,我换成我自己的数据清单就没有问题, 所以建议你仔细检查一下你的数据清单
使用道具 举报 回复 支持 反对
发表于 2021-4-8 20:11:08
zhiyuan 发表于 2021-4-8 17:54
网络上搜不到这个错误,不知道是哪里出了问题,附上代码

另外注意你录制的语音数据是否 是 单声道 44100hz  .wav格式的语音文件
使用道具 举报 回复 支持 反对
发表于 2021-4-8 20:47:55
感谢老师,已发现问题为音频是立体声导致的,已解决,程序正常运行
使用道具 举报 回复 支持 反对
发新帖
您需要登录后才可以回帖 登录 | 立即注册