了解神经网络代码中的混洗和拆分过程

数据挖掘 神经网络 麻木的
2022-03-16 05:08:08

我需要了解以下代码中的标签拆分是如何发生的:

import keras
import librosa
import librosa.feature
import librosa.display
import glob
import numpy as np   
from keras.models import Sequential 
from keras.layers import Dense , Activation
from keras.utils.np_utils import to_categorical


def extract_features_song(f):
    y, _ = librosa.load(f)

    # get mfcc
    mfcc = librosa.feature.mfcc(y)

    # make value between 1 -1 
    mfcc /= np.amax(np.absolute(mfcc))
    return np.ndarray.flatten(mfcc)[:25000]

def extrac_features_and_labels():
    all_features = []
    all_labels = []

    genres = ['blues' , 'classical', 'country' , 'disco' , 'hiphop', 'jazz', 'metal' , 'pop', 'reggae', 'rock']
    for genre in genres:

        sound_files = glob.glob('genres/'+genre+'/*.au')


        print ('prcoessing %d songs in %s genre'% (len(sound_files), genre))

        for f in sound_files:
            features =extract_features_song(f)
            all_features.append(features)
            all_labels.append(genre)

    # one hot encoding
    label_uniq_ids , label_row_ids = np.unique(all_labels, return_inverse= True)
    label_row_ids = label_row_ids.astype(np.int32, copy= False)
    onehot_labels = to_categorical(label_row_ids, len(label_uniq_ids))

    return np.stack(all_features), onehot_labels 

features , labels = extrac_features_and_labels()



print (np.shape(features))
print (np.shape(labels))

training_split = 0.8

alldata = np.column_stack(features , labels)

np.random.shuffle(alldata)
splitidx = int(len(alldata))*training_split
train , test = alldata[:splitidx,:], alldata[splitidx:, :]

print (np.shape(train))
print (np.shape(test))


# the concerned part: begin
train_input = train [:,:-10]
train_labels = train [:,-10:]

test_input = test [:,:-10]
test_labels = test [:,-10:]
#the concerned part: end



print (np.shape(train_input))
print (np.shape(train_labels))

输出如下:

(1000, 25000)
(1000, 10)
(800, 25010)
(200, 25010)
(800, 25000)
(800, 10)

现在,当他 - 代码指导员 - 将两个数组堆叠在一起并打乱它们时,他不能确定最后十个元素是标签,对吗?如果是这样,他是如何做到的,这将导致错误。

1个回答

不,最后 10 个元素肯定是标签。np.column_stack将第二个数组的元素附加到第一个数组的相应行。

一个简单的例子(Python 3.5),随机单热编码片段是从这个答案中复制的

>>> import numpy as np
>>> np.__version__
'1.15.0'
>>> rand_enc = np.random.randint(0,3,size=(10))
>>> labels = np.zeros((10,3))
>>> labels[np.arange(10), rand_enc] = 1
>>> labels
array([[0., 0., 1.],
       [0., 1., 0.],
       [1., 0., 0.],
       [1., 0., 0.],
       [1., 0., 0.],
       [1., 0., 0.],
       [0., 0., 1.],
       [0., 0., 1.],
       [0., 1., 0.],
       [1., 0., 0.]])
>>> features = np.random.rand(10,4)
>>> features
array([[0.51599703, 0.24758895, 0.70951355, 0.52074341],
       [0.64851074, 0.14481191, 0.88837244, 0.86901571],
       [0.36303914, 0.93464881, 0.21238598, 0.24998789],
       [0.74290604, 0.96704141, 0.16971294, 0.99628383],
       [0.20225753, 0.78450864, 0.42730556, 0.84727098],
       [0.95372573, 0.38092036, 0.73628447, 0.22120431],
       [0.10821134, 0.54467407, 0.00452555, 0.15295404],
       [0.7353348 , 0.26629375, 0.96241551, 0.84573258],
       [0.80659848, 0.34873381, 0.12219345, 0.46671669],
       [0.07837654, 0.82128673, 0.79523531, 0.10652154]])
>>> np.column_stack((features, labels))
array([[0.51599703, 0.24758895, 0.70951355, 0.52074341, 0.        ,
        0.        , 1.        ],
       [0.64851074, 0.14481191, 0.88837244, 0.86901571, 0.        ,
        1.        , 0.        ],
       [0.36303914, 0.93464881, 0.21238598, 0.24998789, 1.        ,
        0.        , 0.        ],
       [0.74290604, 0.96704141, 0.16971294, 0.99628383, 1.        ,
        0.        , 0.        ],
       [0.20225753, 0.78450864, 0.42730556, 0.84727098, 1.        ,
        0.        , 0.        ],
       [0.95372573, 0.38092036, 0.73628447, 0.22120431, 1.        ,
        0.        , 0.        ],
       [0.10821134, 0.54467407, 0.00452555, 0.15295404, 0.        ,
        0.        , 1.        ],
       [0.7353348 , 0.26629375, 0.96241551, 0.84573258, 0.        ,
        0.        , 1.        ],
       [0.80659848, 0.34873381, 0.12219345, 0.46671669, 0.        ,
        1.        , 0.        ],
       [0.07837654, 0.82128673, 0.79523531, 0.10652154, 1.        ,
        0.        , 0.        ]])

如您所见,标签被附加在最后!改组代码仅操作行,因此保留了列顺序。您确实需要传递np.column_stack一个 NumPy 数组的元组,否则就可以完成工作。

希望这能说明问题!

编辑:我不知道讲师是如何在 and 之间得到大小不匹配的train_inputtrain_labels从我从您共享的代码中了解到的应该是(800,25000)and(800,10)