我正在嘗試從頭開始實作神經網路。默認情況下，它按我的預期作業，但是，現在我正在嘗試將 L2 正則化添加到我的模型中。為此，我需要更改三種方法-
cost() #計算成本，cost_derivative，backward_prop #向后傳播網路
你可以在下面看到，我有 L2_regularization = None作為init函式的輸入
def __init__(self,sizes, activations = None , cost_function = 'binary_cross_entropy' ,param_init_type = None, L2_regularization=None,dropout = None):
self.sizes = sizes
self.num_layers = len(sizes)
self.caches = dict()
self.cost_function = cost_function
if activations == None: self.layer_activations = self.default_layer_activations_init(sizes)
else: self.layer_activations = activations
if param_init_type == None: self.param_init_type = 'default'
else: self.param_init_type = param_init_type
self.parameters_initializer()
因此，如果 L2_regularization 為 True，我需要對上述方法稍作改動。
我可以復制所有三個功能并更改它們，并在培訓時詢問：
if self.regularization: cost = self.cost_reg(input) # as if i'm overriding the cost function
以及其他人
然而，
這種方式有兩個問題
我不認為這種方式是一種真正的pythonic方式。因此，復制一種方法并為其命名并稍作更改時，它看起來并不好。
I don't want to check whether
self.regularization is Trueorself.regularization is None: in each iteration. I think it could slow down the model and there is a better way instead. If i am wrong let me know about this problem.
What I want from model is to be aware of regularization beforehand.
For example, I have self.regualrization==True
when i call backprop method in train function, it returns back propagation with regularization expression
Code.
It would be too complicated for you to read the whole code and advise me a way of doing things as i want. Hence, i wrote simpler code with same scenario actually
class Network():
def __init__(self,sizes, regularization = None):
self.sizes = sizes
self.expression = 5
self.regularization = regularization
def compute_cost(self):
count = 0
for i in self.sizes:
count =i
return count
def compute_cost_regularized(self):
count = 0
for i in self.sizes:
count =i
#as if self.expression is value of regularization expression
count = count self.expression
return count
def cost_value(self):
if self.regularization:
return self.compute_cost_regularized()
else:
return self.compute_cost()
net_default = Network([3,3,4])
net_regularized= Network([3,3,4],regularization=True)
print('This is the answer from net_default ',net_default.cost_value())
print('This is the answer from net_regularized ',net_regularized.cost_value())
The output is : This is the answer from net_default 10 This is the answer from net_regularized 15
It doesn't solve none of my problems.
I wrote one method 2 times with 1 line change and i used if statement while computing.
如果不這樣做，我怎么能寫出來。我是否需要避免在每次迭代中使用 if 陳述句
我也嘗試覆寫該方法
class Network():
def __init__(self,sizes, regularization = None):
self.sizes = sizes
self.expression = 5
self.regularization = regularization
def compute_cost(self):
count = 0
for i in self.sizes:
count =i
return count
def cost_value(self):
if self.regularization:
return regularized(self).compute_cost()
else:
return self.compute_cost()
class regularized(Network):
def __init__(self, sizes, regularization=None):
super().__init__(sizes, regularization)
def compute_cost(self):
return super().compute_cost() self.expression
net_default = Network([3,3,4])
net_regularized= Network([3,3,4],regularization=True)
print('This is the answer from net_default ',net_default.cost_value())
print('This is the answer from net_regularized ',net_regularized.cost_value())
但是，我收到一個錯誤 TypeError : 'Network' object is not iterable
如果您有實作正則化的修改想法，這里是實際的 train() 和 cost() 函式
def cost(self,X,Y):
#TODO L2 reg ll change cost function
"""param X : Input that will be given to network , Function itself does forward propagation steps and compute cost
param Y : Wanted output corresponds to given input data. Cost will be computed by This Y and Y_hat which is output of NN for X input"""
Y_hat = self.feed_forward(X)
m = Y.shape[1]
if self.cost_function == 'binary_cross_entropy':
cost = (-1/m)*np.sum( np.multiply(Y,np.log(Y_hat)) np.multiply( (1-Y) , np.log(1-Y_hat) )) ; cost = np.squeeze(cost)
return cost
elif self.cost_function == 'mse':
cost = (1/m)*np.sum(np.square(Y-Y_hat)) ; cost = np.squeeze(cost)
return cost
else:
raise Exception('No such cost function yet')
def train(self,X,Y,lr = 0.0001,epoch=1000 , X_test = None , Y_test = None , regularization = None , dropout = False):
assert (X.shape[1] == Y.shape[1]) , "Unmatched In out batch size"
self.caches['A0'] = X
for iter in range(epoch):
A_l = self.feed_forward(X)
dA_l = self.cost_derivative(A_l,Y)
for layer_num in reversed(range(1,self.num_layers)):
grad_w,grad_b,dA_l = self.backward_prop(dA_l,layer_num)
self.update_param(grad_w,grad_b,layer_num, lr = lr)
if iter% (epoch/10) ==0:
print('\n COST:::',self.cost(X,Y),end=' ')
self.score(X,Y)
if X_test is not None:
self.score(X_test,Y_test)
#Saving parameters dictionary to file
a_file = open("parameters.pkl", "wb")
pickle.dump(self.parameters, a_file)
a_file.close()
以防萬一你有興趣，我已經在這里下載了完整的代碼。
https://github.com/IlkinKarimli0/Neural-Network-from-scratch/blob/main/NeuralNetwork.py
uj5u.com熱心網友回復：
一般的
總體而言，您不應該為了覆寫單個方法而在物件內部創建物件，而是可以這樣做
class Network():
def __init__(self, sizes):
self.sizes = sizes
self.expression = 5
def compute_cost(self):
count = 0
for i in self.sizes:
count =i
return count
def cost_value(self):
return self.compute_cost()
class RegularizedNetwork(Network):
def __init__(self, sizes):
super().__init__(sizes)
def compute_cost(self):
return super().compute_cost() self.expression
net_default = Network([3,3,4])
net_regularized= RegularizedNetwork([3,3,4])
print('This is the answer from net_default ',net_default.cost_value())
print('This is the answer from net_regularized ',net_regularized.cost_value())
換句話說，您實際上創建了一個子類的實體，它覆寫了一個特定的函式（這里：compute_cost），并繼承了所有剩余的函式。現在當呼叫 cost_value() 時，它會呼叫相應的 compute_cost。事實上，您也不需要 compute_cost。
class Network():
def __init__(self, sizes):
self.sizes = sizes
self.expression = 5
def cost_value(self):
count = 0
for i in self.sizes:
count =i
return count
class RegularizedNetwork(Network):
def __init__(self, sizes):
super().__init__(sizes)
def cost_value(self):
return super().cost_value() self.expression
net_default = Network([3,3,4])
net_regularized= RegularizedNetwork([3,3,4])
print('This is the answer from net_default ',net_default.cost_value())
print('This is the answer from net_regularized ',net_regularized.cost_value())
代碼問題
如果出于某種原因您想繼續使用自己的代碼，問題是這里
def cost_value(self):
if self.regularization:
return regularized(self).compute_cost()
else:
return self.compute_cost()
you are passing reference to "self" to a constructor of regularized which is expecting sizes, it should be
def cost_value(self):
if self.regularization:
return regularized(self.sizes).compute_cost()
else:
return self.compute_cost()
轉載請註明出處，本文鏈接：https://www.uj5u.com/ruanti/448959.html
上一篇：在JMeter中使用帶有Groovy的JSR223從另一個執行緒組中運行一個執行緒組
下一篇：如何隱藏連接到資料庫URI