当前位置: 主页 > 运维经验 > 运维故障 >

如何解决TIME_WAIT过多的解决办法(附Socket中的T(2)

时间:2013-04-03 23:52来源:转载 作者:网络 点击:
if at all possible. When you close a socket, the server goes into a TIME_WAIT state, just to be really really sure that all the data has gone through. When a socket is closed, both sides agree by send
欢迎大家分享自己的文档,请点击查阅:分享方法,Linux系统运维
如果你喜欢这文章,可以点击文章结尾处百度分享,分享到你的各种社区收藏,或推荐给朋友……



if at all possible. When you close a socket, the server goes into a
TIME_WAIT state, just to be really really sure that all the data has
gone through. When a socket is closed, both sides agree by sending
messages to each other that they will send no more data. This, it
seemed to me was good enough, and after the handshaking is done, the
socket should be closed. The problem is two-fold. First, there is no
way to be sure that the last ack was communicated successfully.
Second, there may be "wandering duplicates" left on the net that must
be dealt with if they are delivered.

Andrew Gierth (andrew@erlenstar.demon.co.uk) helped to explain the
closing sequence in the following usenet posting:

Assume that a connection is in ESTABLISHED state, and the client is
about to do an orderly release. The client's sequence no. is Sc, and
the server's is Ss. Client Server
====== ======
ESTABLISHED ESTABLISHED
(client closes)
ESTABLISHED ESTABLISHED
------->>
FIN_WAIT_1
<<-------- 
FIN_WAIT_2 CLOSE_WAIT
<<-------- (server closes)
LAST_ACK
, ------->>
TIME_WAIT CLOSED
(2*msl elapses...)
CLOSED

Note: the +1 on the sequence numbers is because the FIN counts as one
byte of data. (The above diagram is equivalent to fig. 13 from RFC
793).

Now consider what happens if the last of those packets is dropped in
the network. The client has done with the connection; it has no more
data or control info to send, and never will have. But the server does
not know whether the client received all the data correctly; that's
what the last ACK segment is for. Now the server may or may not care
whether the client got the data, but that is not an issue for TCP; TCP
is a reliable rotocol, and must distinguish between an orderly
connection close where all data is transferred, and a connection abort
where data may or may not have been lost.

So, if that last packet is dropped, the server will retransmit it (it
is, after all, an unacknowledged segment) and will expect to see a
suitable ACK segment in reply. If the client went straight to CLOSED,
the only possible response to that retransmit would be a RST, which
would indicate to the server that data had been lost, when in fact it
had not been.

(Bear in mind that the server's FIN segment may, additionally, contain
data.)

DISCLAIMER: This is my interpretation of the RFCs (I have read all the
TCP-related ones I could find), but I have not attempted to examine
implementation source code or trace actual connections in order to
verify it. I am satisfied that the logic is correct, though.

More commentarty from Vic:

The second issue was addressed by Richard Stevens (rstevens@noao.edu,
author of "Unix Network Programming", see ``1.5 Where can I get source
code for the book [book title]?''). I have put together quotes from
some of his postings and email which explain this. I have brought
together paragraphs from different postings, and have made as few
changes as possible.

From Richard Stevens (rstevens@noao.edu):

If the duration of the TIME_WAIT state were just to handle TCP's full-
duplex close, then the time would be much smaller, and it would be
some function of the current RTO (retransmission timeout), not the MSL
(the packet lifetime).

A couple of points about the TIME_WAIT state.

o The end that sends the first FIN goes into the TIME_WAIT state,
because that is the end that sends the final ACK. If the other
end's FIN is lost, or if the final ACK is lost, having the end that
sends the first FIN maintain state about the connection guarantees
that it has enough information to retransmit the final ACK.

o Realize that TCP sequence numbers wrap around after 2**32 bytes
have been transferred. Assume a connection between A.1500 (host A,
port 1500) and B.2000. During the connection one segment is lost
and retransmitted. But the segment is not really lost, it is held
by some intermediate router and then re-injected into the network.
(This is called a "wandering duplicate".) But in the time between
the packet being lost & retransmitted, and then reappearing, the
connection is closed (without any problems) and then another
connection is established between the same host, same port (that
is, A.1500 and B.2000; this is called another "incarnation" of the
connection). But the sequence numbers chosen for the new
incarnation just happen to overlap with the sequence number of the
wandering duplicate that is about to reappear. (This is indeed
possible, given the way sequence numbers are chosen for TCP
connections.) Bingo, you are about to deliver the data from the
wandering duplicate (the previous incarnation of the connection) to
the new incarnation of the connection. To avoid this, you do not
allow the same incarnation of the connection to be reestablished
until the TIME_WAIT state terminates.

Even the TIME_WAIT state doesn't complete solve the second problem,
given what is called TIME_WAIT assassination. RFC 1337 has more
details.

o The reason that the duration of the TIME_WAIT state is 2*MSL is
that the maximum amount of time a packet can wander around a
network is assumed to be MSL seconds. The factor of 2 is for the
round-trip. The recommended value for MSL is 120 seconds, but
Berkeley-derived implementations normally use 30 seconds instead.
This means a TIME_WAIT delay between 1 and 4 minutes. Solaris 2.x
does indeed use the recommended MSL of 120 seconds.

A wandering duplicate is a packet that appeared to be lost and was
retransmitted. But it wasn't really lost ... some router had
problems, held on to the packet for a while (order of seconds, could
be a minute if the TTL is large enough) and then re-injects the packet
back into the network. But by the time it reappears, the application
that sent it originally has already retransmitted the data contained
in that packet.

Because of these potential problems with TIME_WAIT assassinations, one
should not avoid the TIME_WAIT state by setting the SO_LINGER option
to send an RST instead of the normal TCP connection termination
(FIN/ACK/FIN/ACK). The TIME_WAIT state is there for a reason; it's
your friend and it's there to help you :-)

I have a long discussion of just this topic in my just-released
"TCP/IP Illustrated, Volume 3". The TIME_WAIT state is indeed, one of
the most misunderstood features of TCP.

I'm currently rewriting "Unix Network Programming" (see ``1.5 Where
can I get source code for the book [book title]?''). and will include
lots more on this topic, as it is often confusing and misunderstood.

An additional note from Andrew:

Closing a socket: if SO_LINGER has not been called on a socket, then
close() is not supposed to discard data. This is true on SVR4.2 (and,
apparently, on all non-SVR4 systems) but apparently not on SVR4; the
use of either shutdown() or SO_LINGER seems to be required to
guarantee delivery of all data.

-------------------------------------------------------------------------------------------

讨厌的 Socket TIME_WAIT 问题

netstat -n | awk '/^tcp/ {++state[$NF]} END {for(key in state) print key,"/t",state[key]}'

会得到类似下面的结果,具体数字会有所不同:

LAST_ACK 1
SYN_RECV 14
ESTABLISHED 79
FIN_WAIT1 28
FIN_WAIT2 3
CLOSING 5
TIME_WAIT 1669

状态:描述
CLOSED:无连接是活动的或正在进行
LISTEN:服务器在等待进入呼叫
SYN_RECV:一个连接请求已经到达,等待确认
SYN_SENT:应用已经开始,打开一个连接
ESTABLISHED:正常数据传输状态
FIN_WAIT1:应用说它已经完成
FIN_WAIT2:另一边已同意释放
ITMED_WAIT:等待所有分组死掉
CLOSING:两边同时尝试关闭
TIME_WAIT:另一边已初始化一个释放
LAST_ACK:等待所有分组死掉

也就是说,这条命令可以把当前系统的网络连接状态分类汇总。

下面解释一下为啥要这样写:

一个简单的管道符连接了netstat和awk命令。

------------------------------------------------------------------

每个TCP报文在网络内的最长时间,就称为MSL(Maximum Segment Lifetime),它的作用和IP数据包的TTL类似。

RFC793指出,MSL的值是2分钟,但是在实际的实现中,常用的值有以下三种:30秒,1分钟,2分钟。

注意一个问题,进入TIME_WAIT状态的一般情况下是客户端,大多数服务器端一般执行被动关闭,不会进入TIME_WAIT状态,当在服务

器端关闭某个服务再重新启动时,它是会进入TIME_WAIT状态的。

举例:
1.客户端连接服务器的80服务,这时客户端会启用一个本地的端口访问服务器的80,访问完成后关闭此连接,立刻再次访问服务器的

80,这时客户端会启用另一个本地的端口,而不是刚才使用的那个本地端口。原因就是刚才的那个连接还处于TIME_WAIT状态。
2.客户端连接服务器的80服务,这时服务器关闭80端口,立即再次重启80端口的服务,这时可能不会成功启动,原因也是服务器的连

接还处于TIME_WAIT状态。


检查net.ipv4.tcp_tw当前值,将当前的值更改为1分钟:
[root@aaa1 ~]# sysctl -a|grep net.ipv4.tcp_tw
net.ipv4.tcp_tw_reuse = 0
net.ipv4.tcp_tw_recycle = 0
[root@aaa1 ~]#

vi /etc/sysctl
增加或修改net.ipv4.tcp_tw值:
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1

使内核参数生效:
[root@aaa1 ~]# sysctl -p

[root@aaa1 ~]# sysctl -a|grep net.ipv4.tcp_tw
net.ipv4.tcp_tw_reuse = 1
net.ipv4.tcp_tw_recycle = 1

用netstat再观察正常


这里解决问题的关键是如何能够重复利用time_wait的值,我们可以设置时检查一下time和wait的值
#sysctl -a | grep time | grep wait
net.ipv4.netfilter.ip_conntrack_tcp_timeout_time_wait = 120
net.ipv4.netfilter.ip_conntrack_tcp_timeout_close_wait = 60
net.ipv4.netfilter.ip_conntrack_tcp_timeout_fin_wait = 120

问一下TIME_WAIT有什么问题,是闲置而且内存不回收吗?

是的,这样的现象实际是正常的,有时和访问量大有关,设置这两个参数: reuse是表示是否允许重新应用处于TIME-WAIT状态的

socket用于新的TCP连接; recyse是加速TIME-WAIT sockets回收

 

Q: 我正在写一个unix server程序,不是daemon,经常需要在命令行上重启它,绝大 
多数时候工作正常,但是某些时候会报告"bind: address in use",于是重启失 
败。 

转载请注明linux系统运维
http://www.linuxyw.com/linux/yunweiguzhang/20130403/18.html

------分隔线----------------------------
发表评论
请自觉遵守互联网相关的政策法规,严禁发布色情、暴力、反动的言论。
评价:
验证码: 点击我更换图片
栏目列表
推荐内容