We thought sigmoid is best

[Kyle(120.18)]

We thought sigmoid activation function is best for more than 50 years. Just recently if you draw a straight diagonal line in the plot, we found that it significantly outperforms the other and it is called ReLU. Considering how long we have been doing wrong for half decades on a simple problem, how wrong is the current state of ML?

[☎2.80™(roidz)]

안녕~

[Kyle(120.18)]

안녕

[asdf(104.12)]

This is false. ReLU has its pros and cons.

[Kyle(61.69)]

Uh oh, we have 낄낄 clone nay-sayer

[asdf(104.12)]

Well, saying that doesn't make this true. sigh. can you please google before you make this kind of dumb statement?

[Kyle(61.69)]

It's a statement from Geoffrey Hinton you moron

[asdf(104.12)]

I can hardly believe that's true. Keep saying that to yourself. If you have ever googled on that particular subject, you'd know that's an embarrassing statement.

[Kyle(61.69)]

Sounds like an expert. If you double the number of layers in an NN, how much more time do you expect the backprop to take?

[Kyle(61.69)]

Answer format: times <approximate number>

[Kyle(61.69)]

btw I'm talking about a Deep NN