論文閱讀之Regularized Evolution for Image Classifier Architecture Search

Regularized Evolution for Image Classifier Architecture Search這篇文章實際上解決的是針對NAS這個問題用EA演算法的缺陷，一個很強的motivation是NAS是需要對搜到的網路進行重新訓練的，但是即使是一個比較差的網路，在訓練的程序中也可能因為偶然因素變得很好，如果是傳統的EA演算法，在種群中保留最好的，這樣可能會導致的問題是這個因為偶然性表現比較好的會作為一個精英并且一直延續下去，導致大量的沒有意義的搜索，減少了exploration的能力，作者一個很大的貢獻就是改進了EA演算法，如下所示：

如果是傳統的EA演算法的話，只是移除了最老的，而不是最差的，作者說這樣可能為有上面所說的問題，原文是：

We can speculate that aging may help navigate the training noise in evolutionary experiments, as follows. Noisy training means that models may sometimes reach high accuracy just by luck. In non-aging evolution (NAE, i.e. standard tournament selection), such lucky models may remain in the population for a long time—even for the whole experiment. One lucky model, therefore, can produce many children, cause reducing exploration. Under aging evolution (AE), on the other hand, all models have a short lifespan, so the population is wholly renewed frequently, leading to more diversity and more exploration. In AE, because models die quickly, the only way an architecture can remain in the population for a long time is by being passed down from parent to child through the generations. Each time an architecture is inherited it must be re-trained.

作者的意思比較明了，值得注意的是最后回傳所有val arch上最好的結果，
還有就是，上面這張演算法圖給我們展示了標準的tournament selection的EA演算法是一個怎么樣的流程，比如一開始初始化種群，然后隨機選取多少個，成為tournament size，從中選擇最好的，對其進行變異，然后淘汰最老的或者最差的，
當然作者也試驗了一下他們搜索的結果，對比了一下引入age和不引入age的實驗結果

絕大多數結果都是比較好的