왜 그런건가요?? require grads다 확인햿는데 각 레이어의 grad값 확인해보면


Layer: module.pretrained_model.encoder.encoder.layers.11.final_layer_norm.bias, Gradient: 4.403067919156456e-07

Layer: module.pretrained_model.encoder.encoder.layer_norm.weight, Gradient: -3.830154128081631e-06

Layer: module.pretrained_model.encoder.encoder.layer_norm.bias, Gradient: 1.3824292182107456e-06

Layer: module.pretrained_model.encoder.feature_extractor.conv_layers.0.0.weight, Gradient: 0.0

Layer: module.pretrained_model.encoder.feature_extractor.conv_layers.0.2.weight, Gradient: 0.0

Layer: module.pretrained_model.encoder.feature_extractor.conv_layers.0.2.bias, Gradient: 0.0

Layer: module.pretrained_model.encoder.feature_extractor.conv_layers.1.0.weight, Gradient: 0.0

Layer: module.pretrained_model.encoder.feature_extractor.conv_layers.2.0.weight, Gradient: 0.0

Layer: module.pretrained_model.encoder.feature_extractor.conv_layers.3.0.weight, Gradient: 0.0

Layer: module.pretrained_model.encoder.post_extract_proj.weight, Gradient: 3.535463122261717e-08

Layer: module.pretrained_model.encoder.post_extract_proj.bias, Gradient: -1.5265588444890454e-05

Layer: module.pretrained_model.encoder.conv_pos.pos_conv.0.bias, Gradient: 1.1458541848696768e-06

Layer: module.pretrained_model.encoder.conv_pos.pos_conv.0.weight_g, Gradient: 4.5083586883265525e-05

Layer: module.pretrained_model.encoder.conv_pos.pos_conv.0.weight_v, Gradient: -3.9408273977414865e-08

Layer: module.pretrained_model.encoder.layer_norm.weight, Gradient: -6.008135824231431e-05

Layer: module.pretrained_model.encoder.layer_norm.bias, Gradient: 0.00010069859854411334

Layer: module.pretrained_model.proj.weight, Gradient: -3.0447341487160884e-05

Layer: module.pretrained_model.proj.bias, Gradient: -0.0010494327871128917

Layer: module.conv1d_first.weight, Gradient: 0.0

Layer: module.conv1d_first.bias, Gradient: 0.0

Layer: module.conv1d.weight, Gradient: 0.0

Layer: module.conv1d.bias, Gradient: 0.0


이런 식으로 나오네요 ㅠㅠ