What the error is telling you is that a connection attempt was made, but the session authentication was not provided before SQLNET.INBOUND_CONNECT_TIMEOUT seconds.
As far as adverse effects in the long run, you have a user or process that is unable to connect to the database. So someone is unhappy about the database/application.
————————————————————————————————
In 10gR2, SQLNET.INBOUND_CONNECT_TIMEOUT the parameters were set to have a default of 60 (seconds).
Set the parameters SQLNET.INBOUND_CONNECT_TIMEOUT and INBOUND_CONNECT_TIMEOUT_listenername to 0 (indefinite).
Before setting SQLNET.INBOUND_CONNECT_TIMEOUT, verify that there is not a firewall or Network Address Translation (NAT) between the client and server. Those are common cause for ORA-3136.
————————————————————————————————
The parameter is to be added in sqlnet.ora on the server, so wont require a shutdown.
$ORACLE_HOME/network/admin/sqlnet.ora
and edit it at server side.
Reference: Oracle’s database forum and was provided by Eric Jenkinson: https://forums.oracle.com/forums/thread.jspa?threadID=457356
Monish's DBA Blog 
Monish,
That was very helpful man.
Thanks.
MG.
By: MG on October 31, 2009
at 1:44 AM
Thanks a lot Monish. Many days have i seen this error in my alert.log, but now i took action, found your topic, and understood the problem. I will do the changes right away, and follow it up to see how it goes.
Thanks a bunch!
Mihai
By: Mihai on February 17, 2010
at 3:09 PM