TAGNN代碼解讀

hi各位大佬好，上一篇博文看了下paper，也沒啥子，自己寫又寫不出，哈哈

For Video Recommendation in Deep learning QQ Group 277356808

視頻推薦深度學習加這個群

For Visual in deep learning QQ Group 629530787

視覺深度學習加這個，別加錯

I'm here waiting for you

別加那么多，沒必要，另外，不接受這個網頁的私聊/私信！！！

剛開始就報錯了，啊啊啊啊啊啊啊啊啊啊啊啊啊，人生依舊艱難啊，咋整啊，【也就隨口一說】

1-直接將GNN中的資料拿來用發現出錯了

epoch:  0
/home/xulm1/anaconda3/lib/python3.7/site-packages/torch/optim/lr_scheduler.py:122: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)
start training:  2020-10-30 11:20:35.947314
[0/5839] Loss: 10.6708
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:106: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [8,0,0] Assertion `t >= 0 && t < n_classes` failed.
Traceback (most recent call last):
  File "main.py", line 69, in <module>
    main()
  File "main.py", line 48, in main
    hit, mrr = train_test(model, train_data, test_data)
  File "/data1/xulm1/TAGNN/model.py", line 139, in train_test
    loss.backward()
  File "/home/xulm1/anaconda3/lib/python3.7/site-packages/torch/tensor.py", line 195, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/xulm1/anaconda3/lib/python3.7/site-packages/torch/autograd/__init__.py", line 99, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: CUDA error: device-side assert triggered

搜索發現說是類別數啥的多了或者少了，于是將n_node增加或者改小1后都是同樣的錯誤，WOC，只能細看代碼了，看看資料處理部分相同不，

    train_data = pickle.load(open('./datasets/' + opt.dataset + '/train.txt', 'rb'))
    test_data = pickle.load(open('./datasets/' + opt.dataset + '/test.txt', 'rb'))

#對于sample
>>> len(train_data[0])
1205
>>> len(train_data[1])
1205

>>> len(test_data[0])
99
>>> len(test_data[1])
99

基本上可以確定，訓練集資料是由輸入序列和下一個點擊來構成的“inputs-label”，而對于diginetica資料集我可能之前的資料存盤時改了，

請看下面示例：

>>> train_data[0][-5:]
[[272, 287, 287, 287, 271], [272, 287, 287, 287], [272, 287, 287], [272, 287], [272]]
>>> train_data[1][-5:]
[287, 271, 287, 287, 287]

對于有時間先后順序的點擊序列[272, 287, 287, 287, 271]，可以拆分為4個inputs-label，如下：

[272, 287, 287, 287] ——271

[272, 287, 287]——287

[272, 287]——287

[272]——287

于是我重新加載資料查看即可，如下，n_node=43137【要增加1】，這樣得到正確執行結果

>>> np.min(train_data[1])
1
>>> np.max(train_data[1])
43136

start training:  2020-10-30 14:49:05.805033
[0/13] Loss: 5.7167
[3/13] Loss: 5.7231
[6/13] Loss: 5.7123
[9/13] Loss: 5.7171
[12/13] Loss: 5.7065
	Loss:	74.300
start predicting:  2020-10-30 14:49:06.557334
Best Result:
	Recall@20:	11.1111	MMR@20:	1.6607	Epoch:	0,	0

2-小視頻資料測驗

小視頻資料是資料量比較少的了，可以看看運行時間，user大概40萬，item大概8萬，不過鑒于序列長度比較長（40~50），采用隨機2~8的泊松分布起點index算起，

一個問題來了，訓練的資料里面沒有記錄uid，我去，，，，，，咋整啊，先不記錄，先進行測驗下效果吧，

  File "mymain.py", line 28, in main
    train_data,test_data =dat.load_data()
  File "/data1/xulm1/TAGNN/utils.py", line 163, in load_data
    return (train_seq,train_label) (test_seq,test_label)
TypeError: 'tuple' object is not callable

這個錯誤是代碼中兩個（）之間沒有逗號，忘記了

    input_in = torch.matmul(A[:, :, :A.shape[1]], self.linear_edge_in(hidden)) + self.b_iah
  File "/home/xulm1/anaconda3/lib/python3.7/site-packages/torch/nn/modules/module.py", line 532, in __call__
    result = self.forward(*input, **kwargs)
  File "/home/xulm1/anaconda3/lib/python3.7/site-packages/torch/nn/modules/linear.py", line 87, in forward
    return F.linear(input, self.weight, self.bias)
  File "/home/xulm1/anaconda3/lib/python3.7/site-packages/torch/nn/functional.py", line 1372, in linear
    output = input.matmul(weight.t())
RuntimeError: CUDA error: CUBLAS_STATUS_ALLOC_FAILED when calling `cublasCreate(handle)

這個問題還是類別數不正確，修改即可，

修改后記憶體直接爆了，，，，，，咋整啊，搞不起來啊，

RuntimeError: CUDA out of memory. Tried to allocate 2.68 GiB (GPU 0; 10.73 GiB total capacity; 6.46 GiB already allocated; 763.56 MiB free; 9.14 GiB reserved in total by PyTorch)

明天來看看redis存盤吧，還是說減少下用戶量再試試？10萬用戶試試？試了如下的資料，跑得太慢了吧，

(1140664, 2)
Train Lines: 850791
user numner 58098, test_seq length 58098, test_label length 58098
users , [    0     1     2 ... 58095 58096 58097]
The number of users: 58098
The number of items: 47681
The number of ratings: 1140664
Average actions of users: 19.63
Average actions of items: 23.92
The sparsity of the dataset: 99.958823%
-------------------------------------------------------
epoch:  0
/home/xulm1/anaconda3/lib/python3.7/site-packages/torch/optim/lr_scheduler.py:122: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)
start training:  2020-10-30 20:42:16.552962
[0/8508] Loss: 10.7723

但等來的不是順利執行，而是又報錯了，看來必須從1開始編碼啊，

RuntimeError: CUDA error: device-side assert triggered
/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:106: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [0,0,0] Assertion `t >= 0 && t < n_classes` failed.

編碼沒改，上面的是記憶體不夠，發現還有其他定時任務在跑，又跑了下發現，

/pytorch/aten/src/THCUNN/ClassNLLCriterion.cu:106: void cunn_ClassNLLCriterion_updateOutput_kernel(Dtype *, Dtype *, Dtype *, long *, Dtype *, int, int, int, int, long) [with Dtype = float, Acctype = float]: block: [0,0,0], thread: [2,0,0] Assertion `t >= 0 && t < n_classes` failed.
Traceback (most recent call last):
  File "mymain.py", line 65, in <module>
    main()
  File "mymain.py", line 44, in main
    hit, mrr = train_test(model, train_data, test_data)
  File "/data1/xulm1/TAGNN/model.py", line 139, in train_test
    loss.backward()
  File "/home/xulm1/anaconda3/lib/python3.7/site-packages/torch/tensor.py", line 195, in backward
    torch.autograd.backward(self, gradient, retain_graph, create_graph)
  File "/home/xulm1/anaconda3/lib/python3.7/site-packages/torch/autograd/__init__.py", line 99, in backward
    allow_unreachable=True)  # allow_unreachable flag
RuntimeError: CUDA error: CUBLAS_STATUS_INTERNAL_ERROR when calling `cublasSgemm( handle, opa, opb, m, n, k, &alpha, a, lda, b, ldb, &beta, c, ldc)`

>>> torch.cuda.is_available()
True

必須改編碼了，從1開始，總數增加1為n_node，改后可以正常執行了，但是速度太慢了，然而預測時還是記憶體不夠，原因是訓練的記憶體并沒有釋放，

The number of users: 58098
The number of items: 47681
The number of ratings: 1140664
Average actions of users: 19.63
Average actions of items: 23.92
The sparsity of the dataset: 99.958823%
-------------------------------------------------------
epoch:  0
/home/xulm1/anaconda3/lib/python3.7/site-packages/torch/optim/lr_scheduler.py:122: UserWarning: Detected call of `lr_scheduler.step()` before `optimizer.step()`. In PyTorch 1.1.0 and later, you should call them in the opposite order: `optimizer.step()` before `lr_scheduler.step()`.  Failure to do this will result in PyTorch skipping the first value of the learning rate schedule. See more details at https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate
  "https://pytorch.org/docs/stable/optim.html#how-to-adjust-learning-rate", UserWarning)
start training:  2020-11-01 15:15:19.663547
[0/8503] Loss: 10.7745
[1701/8503] Loss: 6.2509
[3402/8503] Loss: 5.0878
[5103/8503] Loss: 4.9218
[6804/8503] Loss: 4.8871
	Loss:	45659.489
start predicting:  2020-11-01 15:33:33.184119
Traceback (most recent call last):
  File "mymain.py", line 65, in <module>
    main()
  File "mymain.py", line 44, in main
    hit, mrr = train_test(model, train_data, test_data)
  File "/data1/xulm1/TAGNN/model.py", line 151, in train_test
    targets, scores = forward(model, i, test_data)
  File "/data1/xulm1/TAGNN/model.py", line 125, in forward
    return targets, model.compute_scores(seq_hidden, mask)
  File "/data1/xulm1/TAGNN/model.py", line 92, in compute_scores
    scores = torch.sum(a * b, -1)  # b,n
RuntimeError: CUDA out of memory. Tried to allocate 1.78 GiB (GPU 0; 10.73 GiB total capacity; 8.61 GiB already allocated; 751.56 MiB free; 9.15 GiB reserved in total by PyTorch)

人生艱難啊，讓我又想起了GPU的怪事，咋整啊，出現這種怪事，一個問題就是占用其他GPU資源，在GPU嚴重匱乏的情況下這種問題很突出，

先這樣？還是換個服務器試試啊？目前也沒多余的服務器啊，，，，，，，，心累，

下次更吧，缺GPU，1080也可以，

我勸你早點歸去，

你說你不想歸去，只叫我抱著你

悠悠海風輕輕吹，

冷卻了野火堆，

我看見傷心的你，

你說我怎舍得去，

苦態也絕美，

如何止哭，只得輕吻你發邊

讓風繼續吹，

不忍遠離，