Jerky transfer

[Shishyan]

Hello...
I have an ethernet based embedded data acquisition device running on a TCP/IP server. From this I am trying to read data and plot them on the screen in real-time. The problem is, in some PCs they work fine but insome others they seems to be too jerky though all the PCs are of same hardware and software configuration, P4 2.4GHz 256MBRAM running WindowsXP SP2.
Going a little deeper
Withing a worker thread I create my socket using WSASocket(AF_INET, SOCK_STREAM, IPPROTO_TCP, NULL, NULL, WSA_FLAG_OVERLAPPED), set the desired network event using WSAEventSelect then connect to my embedded device using WSAConnect(pParent->m_sockBM, (sockaddr *)&stBMAddress, sizeof(stBMAddress), NULL, NULL, NULL, NULL) then after connecting again selct the desired network events as FD_READ | FD_CLOSE and finally within a WSAEnumNetworkEvent I call WSARecv to receive my data. Even I f the PC application as well as the embedded device remains the same, the latency in receiving data becomes more than 2 - 5 seconds when running on some of the PCs. One more strange thing is that, though the size of the receive buffer is 256 bytes I am receiving more than that in one shot!!! WSARecv returns a number close to 2000 within the BytesRead parameter and strangely there are no lost data, only delayed ones
Please help me
Thank you

[MikeAThon]

...One more strange thing is that, though the size of the receive buffer is 256 bytes I am receiving more than that in one shot!!! WSARecv returns a number close to 2000 within the BytesRead parameter and strangely there are no lost data, only delayed ones

Except for the part where your buffer is only 256 bytes, the remainder of what you describe is exactly the way that TCP is expected to work. It's a stream-based protocol, and operates on a stream of individual bytes. TCP does not send messages. It sends bytes, and the way that it groups the bytes together is entirely up to the stack. Your program cannot control the grouping. It's therefore possible that your sending side calls send() twice with two different buffers of 256 each, and that the receiving side: (1) gets all 512 bytes in a single call to recv(), or (2) gets one single byte in a first call to recv() and the remaining 511 bytes in a second call to recv(), or (3) must call recv() three or more times to get even the very first group of 256 bytes, and the last call to recv() actually gets a few bytes from the second call to send(), or (4) any combination at all.

Your code must be written to expect this behavior. It's part of the TCP design. The only guarantee is that all bytes will be received in order.

Mike

[TheCPUWizard]

Except for one clarification, Mike is right on the money :D :D


TCP does not send messages. It sends bytes


It actually sends packets which range in size (not absolutely directly controlable) from 1 byte to about 1500 bytes.

Your "messages" may be broken apart or combines in any ordered fashion. It is up to that application layer to resolve these "bursts of bytes" back into messages.

[Shishyan]

Thank you very much for the input... though its a bit scary.. because i have no problem re-assembling the received bytes, its only the delay introduced what is eating up my head. But again.. why this has to happen only in some of the PCs I am testing on. Does it mean that I can make changes to the WinSock settings (through registry or something else.. thats what I have heard) so that the stack buffer size is reduced to something more suitable.

[MikeAThon]

On a connected TCP socket, there's very little that you can do to reduce latency to below around 200 milliseconds or so. This amount of latency is caused by a number of TCP algorithms that improve overall network efficiency, such as delayed ACK and Nagle. It's possible to fiddle with these algorithms, but at this stage of your network proficiency, don't do it. First understand how the whole thing works, before you decide that you're smarter than the designers.

UDP is faster and, in contrast to TCP, it is message-based. However, it's not guaranteed arrival, or other guarantees of TCP.

If you see latencies of 2 to 5 seconds, as mentioned in your post, than something is seriously wrong, but I doubt it's TCP. It's probably your use of TCP, or interaction between threads in your code.

Mike

[Mitsukai]

hi i think using UDP is better.

tcp is slower because its one complete string in the buffer. but udp stores in buffer as packets.
also udp doesnt make sure the packets are transfered like tcp does (also decreasing performance), in your case i dont think its neccesery to make sure packetes are arived. because once they are arived they are most likely already outdated.

so isuggest looking into udp protocol

[Shishyan]

Yes Mike.. you are right... better to rethink about my code. If you dont mind, please go through the attachment. I have attached my thread function which does the network stuff and receives incomming data. Still what bothers me is, my code is working fine in 50% of the PCs I have tested it on out of a total of 6 PCs.
Please go through the code

stBMAddr.sin_family = AF_INET;

//Create a socket for operation
if(INVALID_SOCKET == (sockBM = WSASocket(AF_INET, SOCK_STREAM, IPPROTO_TCP, NULL, NULL, WSA_FLAG_OVERLAPPED)))
{
....
}

//Select events to be reported from above socket
if(SOCKET_ERROR == WSAEventSelect(sockBM, haEvents[0], FD_CONNECT))
{
...
}

//Connect
if(SOCKET_ERROR == WSAConnect(sockBM, (sockaddr *)&stBMAddr, sizeof(stBMAddr), NULL, NULL, NULL, NULL))
{
if(WSAGetLastError() != WSAEWOULDBLOCK)
{
...
}
}

//Wait till a connection is made
dwWaitRet = WSAWaitForMultipleEvents(2, haEvents, FALSE, IDU_CBED_CONN_TIMEOUT, FALSE);

if(WAIT_OBJECT_0 != dwWaitRet)
{
...
}

//Enumerate the netwrok event
if(SOCKET_ERROR == WSAEnumNetworkEvents(sockBM, haEvents[0], &stWSANetEvent))
{
...
}

//Now reset the event
WSAResetEvent(haEvents[0]);

//See if it was a connect event
if(FD_CONNECT != stWSANetEvent.lNetworkEvents || NULL != stWSANetEvent.iErrorCode[4])
{
...
}

//Select events to be notified about
if(SOCKET_ERROR == WSAEventSelect(sockBM, haEvents[0], FD_READ | FD_CLOSE))
{
...
}

//Make a bridge request to the acquisition card
byaBuf[IDU_ETH_PKT_OFS_START] = IDU_ETH_PKT_START;
byaBuf[IDU_ETH_PKT_OFS_TYPE] = IDU_ETH_PKT_TYP_BRIDGE;
byaBuf[IDU_ETH_PKT_OFS_LENGTH] = 1;
byaBuf[IDU_ETH_PKT_OFS_DATA] = IDU_ETH_PKT_BRIDGE_B;
stWSABuf.buf = (char *)byaBuf;
stWSABuf.len = 4;

if(SOCKET_ERROR == WSASend(sockBM, &stWSABuf, 1, &dwBytesSent, NULL, &stWSAOv, NULL))
{
if(WSA_IO_PENDING != WSAGetLastError())
{
...
}
}

//Reset the event object
WSAResetEvent(stWSAOv.hEvent);
WSAResetEvent(haEvents[0]);
WSAResetEvent(haEvents[1]);

stWSABuf.buf = pParent->m_wpBuff;
stWSABuf.len = 256;

//Enter the thread loop
while(TRUE)
{
//Wait for either something to be received or the user to quit
dwWaitRet = WSAWaitForMultipleEvents(2, haEvents, FALSE, IDU_CBED_CONN_TIMEOUT, FALSE);

if(WAIT_OBJECT_0 == dwWaitRet)
{
//Enumerate the network event
if(SOCKET_ERROR == WSAEnumNetworkEvents(sockBM, haEvents[0], &stWSANetEvent))
{
...
}

//Reset the event object
WSAResetEvent(haEvents[0]);

//Find out what had happened
if(FD_READ == stWSANetEvent.lNetworkEvents)
{
dwFlags = NULL;

//Reset the event object
WSAResetEvent(stWSAOv.hEvent);

//Read in the received data
if(SOCKET_ERROR == WSARecv(sockBM, &stWSABuf, 1, &dwBytesRecvd, &dwFlags, &stWSAOv, NULL))
{
if(WSA_IO_PENDING != WSAGetLastError())
{
...
}

//Wait for IO to complete
if(WAIT_OBJECT_0 != WSAWaitForMultipleEvents(1, &stWSAOv.hEvent, FALSE, IDU_CBED_CONN_TIMEOUT, FALSE))
{
...
}
}

//Call decoder function to parse the received data
...

}else
{
...
}
}else if((WAIT_OBJECT_0 + 1) == dwWaitRet)
{
...
}else if(WAIT_TIMEOUT == dwWaitRet)
{
...
}else
{
...
}
}

return 1;
}


thank you

[MikeAThon]

There's too much code, and too much left out. For that amount of code, it's better to zip up your entire project and post the project, so there are no questions.

In the little that I saw, I noticed one error caused by a misunderstanding of the network event: your code must expect that more than one network event can be set at a time. It's therefore wrong to check for equality against FD_READ (for example), and it's wrong to test in an if-elseif tree. IOW, this is wrong:
// wrong, don't do this
if(FD_READ == stWSANetEvent.lNetworkEvents)
{
// do read stuff
}
else if(FD_WRITE == stWSANetEvent.lNetworkEvents)
{
// do write stuff
}
else if(FD_CLOSE == stWSANetEvent.lNetworkEvents)
{
// do close stuff
}


This is wrong because the equalities will fail if more than one event is set.

Instead, because more than one network event can be set at any one time, your code should look like this (note use of bit-wise AND and elimination of "else"):
// better
if(FD_READ & stWSANetEvent.lNetworkEvents)
{
// do read stuff
}
if(FD_WRITE & stWSANetEvent.lNetworkEvents)
{
// do write stuff
}
if(FD_CLOSE & stWSANetEvent.lNetworkEvents)
{
// do close stuff
}


Mike

PS: Please use [ code ] [ /code ] tags, as I have done above, to make your code more readable