Help Needed: One "recv()" contains data sent by many "send()" ...

[VipulPathak]

Sub: Help Needed: One "recv()" contains data sent by many "send()" ...
Hi,

I have a Server Application, as usual listening on certain port and
accepting connections. It recvand send data between clients. I have a fixed format of sending messages from clients. All clients send data enclosed in*[ and ]* character sequence, eg: suppose the data from client A is this -
*[Hello, How are you? Sending filename in next message]*
then next message will be-
*[C:\VipulPathak\SendableFiles\RunTimeLog.csv]*

Now, this works fine. But when any client start sending data too rapidly, then it often become possible that 2 messages clubbed into one recv(), for example, On executing recv() with a buffer of size > 512 bytes, the buffer possible might contain this:
*[Hello, How are you? Sending filename in next message]**[C:\VipulPathak\SendableFiles\RunTimeLog.csv]*...

Only one recv now contain both the messages. In this situation, I need to parse data and separate the messages manually by string processing. But possible many times more messages behind these messages. And usually the last messages is Incomplete.

My concern is this Incomplete message, in my application these Incomplete messages are usless. So I must eleminate them. I can see 2 options to solve, this problem:
a) Either I receive one message at a time from WinSock Layer, (ie One recv() call must return one message).
b) OR the Last message must be complete, if there is space but not as much that can hold next message, then this left space must not be used.

Am I right, any other Solution some one can suggest.
Also, please suggest a way, using which atleast one of the option a -or- b, can be achieved.

Regards.

-Vipul Pathak ;

[Bob Sheep]

You must do one of these 2 options

1 - Peek at the data and determine the end of the first complete message and then recv only that amount of data.

[This is my message][incomplete...

detect the closing ] at byte 20 and recv only 20 bytes

or

2 - stream the data your self by recv all the data, use only the complete message leving [incomplete... in your buffer and recv the next block of data starting at offset 15 in this case to add to the data you already have. If you see a ] then process the message leaving incomplete data in the stream buffer.

I'll leave how to implement this as an exercise for you. Too many people just ask for "cut-and-pastable" code in this forum.

[DeepButi]

VipulPathak,
as far as I know this is IP protocol behaviour. It tries to minimize number of sent messages, so several logical messages go into one physical message (separated by hex 0, you can check for it if you need to parse).

You must deal with it as Bob Sheep suggests, there is no way to tell IP not to do it because it's done outside your control.

My typical two solutions are:
a) If you get incomplete messages, simply increase your receiving buffer size.
b) If you can control the client, wait a reasonable amount of time between sends. Obviously this is app dependent ... 200ms? 300ms? is acceptable to you?

Hope it helps

[TheCPUWizard]

As posted previously, there is NO correlation between the information in a specific Send and a Specific Recv. The ONLY thing that is guarenteed is tht the bytes will arrive in the same order [or an error will occur!].

It is completely your responsibility to handle sends that are split into multiple recv's and multiple sends that are combined into a single recv, plus any combination of the previous.

[VipulPathak]

Hi,

Thanks for all your replies. I am not asking for ready to use source code, just asking for expert suggestions. Thanks for the Replies.

As all of you have suggested, I should either first peek the recv, then find the closing "]*" and then read only this much of bytes. The situation at my end is, that I am using IO Completion Ports for socket IO, and when any Recv() request completes, It give me all the data it has received in Handle in a buffer. so I don't have choice to peek and then read partially. The other scheme however can work.

Any comments on this Please ...
Regards.

-Vipul Pathak ;

[Bob Sheep]

Here's a stream buffer class I ripped out of some of my code. It might need some tidying up as it was in a dll.

[TheCPUWizard]

Did not look at sheep's example (and they are usually very good), but wanted to offer a generic suggestion.

When using completion ports, you can buffer the data as it is recieved and parse it withing the context of the completion post. When you detect that ou have a complete message you can then take the appropriate action.

A common implementation is to use a queue and an event.

Remember you must handle concatenated and borken messages.

[Mathew Joy]

If someones post make sense, it is TheCPUWizard's.

Firstly...DO NOT USE PEEKING. Peeking is bad. It is buggy, inefficient and unreliable.

Secondly...The said problem is not the IP protocol's behavior. It is the TCP's behavior

Thirdly...By increasing the buffer, you may not get the complete messages. In other words, getting partial messages has nothing (much) to do with recv buffer

Fourthly...Having an interval between sends is also bad. One, it is not guaranteed to succeed. Second, you loose throughput.

One send doesn't mean one recv. TCP is a stream based protocol so it doesn't preserver message boundaries. To comment on what method to use, you need to provide the load, the max concurrent connection, expected response time, resource the server is having... But the method you follow may not that that good because you have to scan each char to determine a complete message. Rather it is good to put a message length just before the message. (this may not be that imp if it is a low load server.)

A common (best) way is to read (drain) the buffer as quick as possible. Then process the buffer there. "Draining" the buffer is important because leaving it in the tcp buffers uses up a vital resource (Non-paged memory).

In your post you said you are using iocp. Since it is a complex design, it is necessary only if you are programming a high performance server.

As said, we may be able to comment more on the design if you could give a clear pict of your server

/consequence of not posting in the specific forum.

[VipulPathak]

Hello Mathew,

As asked by you, Let me briefly describe the Requirements of the Server module.

Server must be able to maintain connections between around any thing between 100-5000 simultaneous clients. The messages sent by these clients will be sent to another process, for saving either on disc or in Memory (depending on the priority tag). Some other processes in the solution, will put the reply of those messages at the same Repository process. Our Server will then read back the messages, and send it to appropriate client. To find out which reply message should be forwarded to which connected client, the server parse the reply-message string and get the NC_Id (which is a 4 char unique string).

Since our Server maintains a <List> (soon going to be replaced by a Hash map), of NC_ID v/s. the Connection related Information for each client, When ever a reply is detected, we first scan the List for NC_ID and get its connection Information. Connection Information includes: SOCKET, Ptr to OVERLAPPED Structure, NC_ID, A unique buffer acquired using new etc.. When a search succeed, we get this information, using which we send the data to client on its socket.

Since our requirement is that the client application may send data rapidly (< 5% chance), we need a fast server. coz. at some extent, if all connected clients (Avg. 2500) start sending data quickly, then the server must not get Clogged. This was the only reason, I switched to IOCP mechanism. Now, I am getting cancatinated messages.

As now it is clear to me, that Its the behaviour of TCP (Stream based), and this was likely to happen. I just want to know, what should i do, in case of receipt of incomplete message. Should I add one more <String> field in the Connection Information, that will store the Half string received, (waiting of another half of the message) or some more Innovation Idea is there ....

Thanks to all Experts in the thread for there suggestion as well ...

Regards.

Vipul Pathak ;

[Mathew Joy]

Good. Now we are getting somewhere. Some info to comment on.
Originally posted by VipulPathak
Hello Mathew,

As asked by you, Let me briefly describe the Requirements of the Server module.

Server must be able to maintain connections between around any thing between 100-5000 simultaneous clients.Yea...IOCP can easily handle 5000 clients, provided you do it correctly. The messages sent by these clients will be sent to another process, for saving either on disc or in Memory (depending on the priority tag). Some other processes in the solution, will put the reply of those messages at the same Repository process. Our Server will then read back the messages, and send it to appropriate client. To find out which reply message should be forwarded to which connected client, the server parse the reply-message string and get the NC_Id (which is a 4 char unique string).Hmm... seems to be OK. But it'll be better if you could packetize the data. Somethis like, ID,REQUEST,DATA_LEN will be the header, and the data till the len specified will be the our required data. These can be a structure. You can readoff the structure from the buffer so the alignment will never be missed.

Since our Server maintains a <List> (soon going to be replaced by a Hash map), of NC_ID v/s. the Connection related Information for each client, When ever a reply is detected, we first scan the List for NC_ID and get its connection Information. Connection Information includes: SOCKET, Ptr to OVERLAPPED Structure, NC_ID, A unique buffer acquired using new etc.. When a search succeed, we get this information, using which we send the data to client on its socket. I didn't get you quite clearly, but you don't have to store those things to a list. You can associate pointers to 2 kinds of data when you associate the socket to the completion port and when you call a WSARecv(). And you get those when you call GQCS() in a thread. One is the completion key and the other is the overlapped pointer. So to one structure associate all the data pertaining to a socket handle (that'll be the completionkey) and other pertaining to the IO (that'll be the overlapped structure).

Since our requirement is that the client application may send data rapidly (< 5% chance), we need a fast server. coz. at some extent, if all connected clients (Avg. 2500) start sending data quickly, then the server must not get Clogged.Actually there can be two types of fast servers. One is fast in terms of connections and the other is fast in terms of data thruput. I assume you are talking of the second. Let me give you a few tips.

You recv/send buffer must be multiples of 4k becase 4k is the page size. When you do a recv/send the entire page is locked up so even if you are using 1byte buffer you are locking 4k memory threby wasting 4k-1 bytes!
For the server to be responsive you should have as much as overlapped outstanding io as possible
If you've read from somewhere that disabling SO_RECVBUF and SO_SNDBUF will improve performance it is wrong. Just leave it as it is.

Said that you must be careful about the second point. How much overlapped io to issue depends on the RAM you are having (to an extend free RAM). You didn't tell about the resource the system is having (No. of processers, RAM etc).


This was the only reason, I switched to IOCP mechanism. And you did the right thing.:)

As now it is clear to me, that Its the behaviour of TCP (Stream based), and this was likely to happen. I just want to know, what should i do, in case of receipt of incomplete message. Should I add one more <String> field in the Connection Information, that will store the Half string received, (waiting of another half of the message) or some more Innovation Idea is there ....See...as I told in my previous post, you system should drain the buffer with as much overlapped recvs as possible. If your clients send data in spurts once you recv data you know more data is following...so read and store it to a buffer and later copy it to the struct as I told.

Hope it is clear to you


Additionaly you can search "Write Scalable Winsock Apps Using Completion Ports" article in MSDN for more info

Some of the techniques are also explained in the networking forum. You can do a search
:thumb:

[Bob Sheep]

How do you ensure that the 4K buffer is page aligned? If it's not then you would lock up 2 x 4K pages.

[Mathew Joy]

Originally posted by Bob Sheep
How do you ensure that the 4K buffer is page aligned? If it's not then you would lock up 2 x 4K pages. Well, it has somethig to do with the system architecture. The 4k I said is on the basis of x86 architecture(on which most of the windows run). Under this architecture, when a system locks memories, it does so on page boundaries. So we only have to be concerned on where it ends. In other words, if we have a 4k buffer that is locked, we utilize only one page (completely).

[Bob Sheep]

Yes but how do you get the memory to be allocated AT THE START of the page boundary. If it's allocated half the way into a page then that entire page will be locked and also the next page which is required for the other half of the 4K memory.

[Mathew Joy]

Bob,
When you read, pay close attention on what you read.

Originally posted by Mathew Joy
Under this architecture, when a system locks memories, it does so on page boundaries. It means it'll never allocate memory half way into a page.

[Mathew Joy]

To be clear, if you are confused with memory allocation, malloc() is implemented with page align especially with large blocks.

[Bob Sheep]

Cough, cough.

I did read what you had written. You said memory is locked ON PAGE BOUNDARIES. I was not asking about LOCKING but ALLOCATION.

:thumbd:

[usmarine]

Listen to Mathew Joy. I agree completely with him.

I cannot stress enough about sending a message header in every message which is sent over a socket. Like Mathew said this can be done easily using structures. You can easily combine several different message types by using unions with structures. Your header will determine which type of message is being sent.

[Mathew Joy]

Originally posted by usmarine
You can easily combine several different message types by using unions with structures. Your header will determine which type of message is being sent. Unions and structs. Good point. I intentionally left it out to make the matter simple.

To Bob:
allocation is what I guess I said in my previous post.

/ahh...the difficulty I had to face in making a guy understand about union and struct :rolleyes: (in NW forum)

[Bob Sheep]

Found it. The meat and potatoes are contained in VirtualAlloc.
I was unaware of this allocation algoritm but now I know.

:D

[Andreas Masur]

[Moved thread]

[VipulPathak]

Hello Mathew,

Thanks for the Tips.

Originally posted by Mathew Joy

Actually there can be two types of fast servers. One is fast in terms of connections and the other is fast in terms of data thruput. I assume you are talking of the second. Let me give you a few tips.


What are Fast Servers in terms of Connections ? How to make a connections Quicker ...?

Regards.

-Vipul Pathak ;

[Mathew Joy]

Originally posted by VipulPathak
What are Fast Servers in terms of Connections ?
Generaly speaking, high performance servers should be either of the two categories. High connectivity or high throughput. The reason is to get maximum out of the limited resources. For ex. web servers are high connectivity servers and FTP servers are example of high throughput servers. High connectivity servers are oriented towards max connections (such as 30,000 or 40,000),and not keen to attain high throughput. If you don't program this in mind your server may show out-of-resource after say 10,000 connections (or maybe less) or may even crash if some other drivers don't behave properly when the resource is exhausted.

How to make a connections Quicker ...?The idea is to accept connections faster and to minimize the overhead in making other overlapped requests.

The first can be achieved if you have a separate thread for issueing multiple overlapped accept(). The only API that allows to make overlapped accept is the AcceptEx() API. This API is also meant for performance. Because of its high performance nature, it avoids the expensive socket creation. That means you should have a pool of sockets ready to accept connections. Moreover, if this API is used the connected sockets requires lesser resource (in terms of NP memory)

The latter can be achieved with a '0-buffer trick'. This is to issue overlapped recv with 0 buffer. Since there is no buffer there won't be any locking (hence reducing the resource used). The outstanding call will be satisfied once the data arrives. Then you can issue simple non-blocking recv, to get all the data, till you recv WSAEWOULDBLOCK error.

Another important thing for performance intrinsic servers is, it should monitor all the o/s calls and decrease them if you see the server is running low on resource (i.e. using too much of resource). And if you see such things happening your server should be able to recover than simply allowing the server to die. You might have to close 'expensive'(interms of o/s calls) connections, make some of them stale, wait till you issue overlapped calls, not allowing overlapped calls etc. One of the best way is to simulate the high load situation and monitor how your server is performing/behaving and limit accordingly.

Hope it is clear!

Good luck

:thumb:

[VipulPathak]

Originally posted by Mathew Joy

The latter can be achieved with a '0-buffer trick'. This is to issue overlapped recv with 0 buffer. Since there is no buffer there won't be any locking (hence reducing the resource used). The outstanding call will be satisfied once the data arrives. Then you can issue simple non-blocking recv, to get all the data, till you recv WSAEWOULDBLOCK error.


Thats Really Great !! :-)

Actually My Application is slow in terms of Connection speed. When I test is Issue so many Connections from the test application, I looks like moving ahead slowly ... :-(

Thats why asked .... Thanks any way ... :-)

Pls. confirm that by Zero-Buffer you mean, In WSARecv() in need to set "dwBufferCount" to ZERO, right ? or The LPWSABUF must be NULL ?

Regards.

-Vipul Pathak ;

[Mathew Joy]

Hi Vipul,
Zero buffer recv means that the length of the buffer should be 0. Doesn't necessarily mean that the buffer should be NULL. In other-words the len parameter of the WSABUF should be 0.

This trick is necessary only for high connectivity server and may actually decrease performance in terms of throughput, because your 0 buff recv is doing nothing (in the sense of getting data) and we are actually using an extra call. This trick is to reduce the no. of locked pages that we use, which is approx 1/8th the total RAM.

As I said, your server should either be high interms of connection or throughput. If you try to make it both, your server may not be able to use the resources efficiently and may end up crashing. Depending on your server (load) AcceptEx can handle somewhere between 2000 to 4000 connections per second (should handle atleast 1500).

:thumb:

[VipulPathak]

Thanks Mathew for the Important Concepts.
Thanks a lot.

Regards.

-Vipul Pathak ;