Problem with slow send() on a local TCP network

[emericbart]

Hi

I'm currently developping, on Windows, a TCP client that connects to a local host (that I didn't develop), and sends/receives data to/from it. Everything works fine, but the problem is it's very slow, especially sending data.

My code works fine, and apparently this looks like a pure network problem. I think i've created and connected my socket correctly, so here is my question : is there any special thing to do to set correctly a local network ? For example, any special option with setsockopt ? Or any TCP window size to change ?

I read that TCP could slow down the send() data rate not to overload the application, is there a way to avoid that (my client would not be overloaded at all if it went faster) ?

Thanks you for any help.

[ahoodin]

1. Using BLOCKING or NON-BLOCKING sockets?

2. How much data are you sending? What sizes are the databuffers on the client?

3. Do other previously written clients for this system work?

4. How many simultaneously connected sockets are there at any one time?

5. What are ping times?

6. Have you tested the sockets data frequency response?

7. Have you stepped through the code with the debugger?

[MikeAThon]

Plus:

When you say "local host", do you mean that both applications are running simultaneously on the same machine? Or is your client running on a machine different from that of the "host", communicating over a LAN?

How slow is it, as compared to 10/100 mbits/sec speed of typical LAN?

Are you making a new connection for each send, or are you re-using a single connection for all sends?

Is each send() sending tiny little data chunks, or are you coalescing all your data into a few large buffers and calling send() only a few times with large chunks of data?

Are other programs able to achieve good transfer rates? What about a simple file copy?

Mike

[emericbart]

Sorry, I'm not very used to posting, especially about network, so I forgot essential things.

So:

1. Blocking sockets.

2. I send 352*288*4 bytes buffers.

3. It's the first client I write for this application.

4. 2 sockets are connected at the same time.

5. Both client and host are on the same computer, so ping times are 0.

6. How can I do that ?

7. Well of course, i've been doing that all day long.

And:

By "local" I mean that both client and host are running on the same machine.

The speed is about 5-6 mbits/sec.

I use a single connection for all sends.

I send big buffers (352*288*4 bytes).


Thank you for your help...

[ahoodin]

OK.

This will result in up to 264 calls to send for each buffer, taking an average of 200ms each if they are sent on a network(due to the nagle algorithm and frame buffer sizes). There is a 1500 Byte Frame Buffer on 10/100 Mbit Network. Gigabit would be 9000 Byte.

The loopback address will be 127.0.0.1. Using the loopback address should decrease your send times by not transmitting the packets on the network.

It will help you to use:

1. Multiple Sockets.
2. Asynchronous communications.

Tangentsoft has great sample code examples of these:
http://tangentsoft.net/wskfaq/

[MikeAThon]

Just to make certain that I understand the basic set-up:

One single machine has both client and host, so your TCP communications never even leave the machine, or reach a LAN.

Your client is constantly connected, using the same two sockets, for the entire time

Your client allocates a buffer of size 352*288*4=405504 bytes. It sends the entire buffer in one single (blocking) call to send().

Your client does that repeatedly, and you still only see a datat transfer rate of 5-6 mbits/sec.

If all of the above is correct, then that's a very low data transfer rate.

In my experience, such low rates are usually not caused by TCP, so tuning TCP window size etc. will not help.

The problem is usually caused by application-level handshaking between the client and the host/server. For example, according to your protocol, does your client wait for an acknowledgement receipt from the host after sending a first 405504 byte buffer, before sending a second 405504 byte buffer? In other words, from the perspective of the client, is the protocol something like (1) block in send() until all 405504 bytes have been sent, (2) block in recv() until the host acknowledges receipt, (3) repeat for the next buffer of 405504 bytes.

Any kind of handshaking will slow data transfer significantly. Tell us more about the application-level protocol for communication between the host and the client.

Mike

[emericbart]

Yes, that's exactly how it works, so that's indeed a very low transfer rate.

About handshaking, i don't wait for anything from the host when i send actually. Send() never blocks (i mean i don't wait for the whole data to be sent before going on, i just call it and go), and recv() blocks until there is something to receive (like on a usual blocking socket), but this is on a seperate thread not to slow down my application.

Here is a description of one of the problems i saw while debugging, maybe that would be helpful:
When I call recv and loop on it to get the whole buffer, the send() from the host is so slow that sometimes the recv() blocks in the middle of the transfer because data is still not here. This results in a recv loop that lasts up to 150 ms to get my whole 405504 bytes buffer.
That's what makes me say it's still a network problem, but maybe I'm misinterpreting this.

Also, by seeing code in the link ahoodin gave me, i would like to know : i don't use those events things, but my communications are asynchronous in the sense that the recv is done on a separate thread (simply because I know better how to do this). Would using those events change something?

Thank you for your help.

[emericbart]

I don't know if that's useful, but here is another thing about that recv() loop problem.
If I don't wait between 2 recv() calls (in the same receiving loop), a single recv only reads small size (usually 2048 bytes), and sometimes waits before beeing able to read it. The more I wait between 2 recv, the more the next one can read, without blocking.
Therefore, by waiting, I can can get my whole buffer in 4-5 recv, and if I don't wait, it takes more than 100 recv().

[MikeAThon]

You're description is not completely clear, because you do not distinguish between client and host.

Which one is waiting between recv()'s, the client or the host? How is the wait accomplished? Sleep()?

From your original post, I thought the problem was a slow send() from client to host. Now it seems like the problem is a slow recv() from the host? Which is it? Or both???

Also, I got the impression that source code for the host was not available. Is it? And are you able to run both the client and the host in the debugger? And make changes tot the host's code?

Mike

[MikeAThon]

Incidentally, in direct answer to your question about TCP settings, look at setsockopt() and the SO_RCVBUF and SO_SNDBUF options. The default Windows settings for these are 8k each (16k in some systems) which should be plenty under most circumstances, including the local-host-type transfer that you are trying to accomplish. But for a lark, try increasing them, on both sides, to 405K.

One last thought: try turning off all anti-spyware software, antivirus software, and firewalls. Additionally, scan your system for spyware. All four categories of software usually work by inspecting the data in the TCP stack. It might be this inspection that is slowing down your transfer.

Mike

[emericbart]

I thought my description would not be very clear, in fact. Sorry.

So, the recv() I'm talking about is on the client, and I wait by using Sleep(). I deduced from these tests that this was a slow send() problem, because whatever the size the recv() manages to read, it is very fast. The problem is it sometimes blocks because data sent by the host is not available fast enough.

And I don't have the host's code, an don't debug on it. All these tests are done on the client.

Also, concerning your remark about antivirus and so, I have a question, maybe stupid because I don't know very much about it, but I ask anyway : can a proxy cause the same problem (I mean scanning the TCP stack) ? Because I'm behind a proxy (on a company network).

Thank you again for your help.