*** Reason for rollback ***

Correctness > speed. And using log1p improves accuracy significantly, especially of gradients.

*** Original change description ***

Partial rollback of change to softplus functor. The large speedup observed was only true for arguments greather than ~5. For arguments less than ~5, a significant slowdown would occur.

PiperOrigin-RevId: 309499644
Change-Id: Ie137dd5e2fa798f8c1e0330375e3e0b6dbcc31a6

tensorflow/core/kernels/softplus_op.h

@@ -53,7 +53,7 @@ struct Softplus {
     activations.device(d) = too_large.select(
         features,                       // softplus(x) ~= x for x large
         too_small.select(features_exp,  // softplus(x) ~= exp(x) for x small
-                         (features_exp + features.constant(T(1))).log()));
+                         features_exp.log1p()));
   }
 };