recv() on HTTP

[Payne747]

Hi all,

I'm new so be kind... ;)

Wondering if anyone can give me tips\advice for C++ programming with the HTTP protocol - specifically using recv(). Basically I call recv() to get the server response, but with a persistent connection, the server never returns 0.

I'm guessing it's not possible with blocking sockets (yes... I'm still starting out!), so once the server returns the HTTP response, I'm still blocking on recv() because the server is keeping the connection open.

My question is, is it possible to use blocking sockets and still determine how much of the HTTP response is left to read (I know content-length will tell me, but my first recv() call has no way of knowing how much of the HTTP response body it already contains, unless I search for \r\n\r\n and count from there...?)

Sorry if this doesn't make much sense, even so - if anyone has any good tips for reading an entire HTTP response into a buffer, I'd appreicate it :)

Lastly, for flattery, first impressions and all - you guys have already helped me on many occasions in the past ;)

Thanks,

Payne747

[skval]

HTTP request always ends with \r\n\r\n.
If you don't want to perform cmplicated string routines,
You can set timeout to the socket by using setsockopt() (http://msdn.microsoft.com/en-us/library/ms740476%28VS.85%29.aspx).
You can also use WSARecv- which returns immediately and set up an event when information was recieved.

[DreamShore]

Where did you get the idea that recv will block after the server responses, anyway? Non-blocking socket is usually used only on server-side, to reduce the excessive and unnecessary number of threads.

Determining the size of http packet is a horrible thing, though. In my word http is something "the size, which is counted from a undetermined position, is itself stored at a undetermined position with undetermined length" To know the size, you have to find the Content-Length item in the header. And then you need find the end of the header, that is the "\r\n\r\n". From there... after the bytes number you retrieved from Content-Length, is the end of the packet.

For keep-alive connection, no Content-Length have to mean there is no more data other than the header.

What you want is merely finding match in a stream, nothing quite special.

[Payne747]

Thanks for the responses, the problem is that I loop on recv(), with an exit condition being it returns 0, like this:

while ((len = master_sock.recv(recv_buf)) > 0) {
// check for valid HTTP, and look for \r\n\r\n
}

When the server finishes sending the HTTP response, it stays open, and my recv() call never ends.

I thought about processing 1 byte at a time in recv_buf, so I know exactly when I reach \r\n\r\n, but can't help thinking this is *really* bad idea.

I want to stick to best practices I've read about, receiving everything in TCP buffer at once, then processing it and requesting more if needed, but with HTTP this is stupidly hard :(

Guess it's back to pulling wget apart for me ;)

[hoxsiew]

I thought about processing 1 byte at a time in recv_buf, so I know exactly when I reach \r\n\r\n, but can't help thinking this is *really* bad idea.


I don't see why you would think this is a bad idea. It's not like there's a full TCP packet sent for each char. Most of the sent data is probably already in the TCP/IP stack buffers anyway, so reading it a char at a time is going to require a minimal amount of overhead.

[DreamShore]

It is just the loop will never ends. Actually it will end when the client side close the connection.

Processing a stream doesn't mean in the most basic way. When you are finding a string in the stream, you can keep the last string_length - 1 characters before refilling the buffer with new data(This is still a very basic and silly way though, but much more efficient than reading a byte each time).