Post Reply 
 
Thread Rating:
  • 0 Votes - 0 Average
  • 1
  • 2
  • 3
  • 4
  • 5
Http looping on 416
23-11-2016, 07:47 AM (This post was last modified: 23-11-2016 08:00 AM by medoc92.)
Post: #1
Http looping on 416
Hi,

I am currently developping a gateway between a streaming service and ohPlayer.

The server does not support range requests. So it just serves the data without giving any indication that it supports ranges (no Accept-Ranges, no Content-Range headers). I see and ignore the initial range spec: 0-

After serving the data, the server gets a range request anyway (for range currentdatasize-), to which it answers with a 416 and no data.

At this point, ohPlayer loops on this same request. I have tried other response codes with the same result.

I am testing with the binary version of ohPipeline on a raspberry pi.

When looking at the git code in ProtocolHttp.cpp line 481 and Protocol.cpp ProtocolManager::Get (line 473), it does appear that the reader should heed the 416, but I did not look in the upper layers, maybe the looping occurs there.

Is the binary code up to date from the current ohPipeline git code ? And if yes, what can I do to make it accept that this request is indeed done finished EOF please desist ?

I tried to build ohpipipeline from git, with no success.

On my side, the very same http code works with every other renderer it has been thrown at at this point.

For completeness, if I kill the server process, while the player is looping on its range request, it goes on busy-looping on:

>ProtocolNetwork::Connect
ProtocolNetwork::Open
ProtocolNetwork::Close
<ProtocolNetwork::Connect error connecting
ProtocolHttp::WriteRequest Connection failure
Find all posts by this user
Quote this message in a reply
23-11-2016, 01:52 PM (This post was last modified: 23-11-2016 01:52 PM by simonc.)
Post: #2
RE: Http looping on 416
(23-11-2016 07:47 AM)medoc92 Wrote:  I am currently developping a gateway between a streaming service and ohPlayer.

The server does not support range requests. So it just serves the data without giving any indication that it supports ranges (no Accept-Ranges, no Content-Range headers). I see and ignore the initial range spec: 0-

After serving the data, the server gets a range request anyway (for range currentdatasize-), to which it answers with a 416 and no data.

At this point, ohPlayer loops on this same request. I have tried other response codes with the same result.

What HTTP headers does your server return? I'm particularly interested in the HTTP version it uses (1.0 or 1.1) and whether it reports a content length, uses chunked transfer encoding or just writes unspecified amounts of data then closes the connection.

If your server does report a length, are that many bytes transferred?

(23-11-2016 07:47 AM)medoc92 Wrote:  Is the binary code up to date from the current ohPipeline git code ? And if yes, what can I do to make it accept that this request is indeed done finished EOF please desist ?

The ohPlayer binaries are a couple of months behind ohPipeline code. This shouldn't affect your issue however - ProtocolHttp hasn't had any interesting changes in that time.

(23-11-2016 07:47 AM)medoc92 Wrote:  I tried to build ohpipipeline from git, with no success.

Have you followed the instructions in the README ? If you had errors during dependency fetching, try fetching dependencies one at a time. E.g.
Code:
./go fetch ohWafHelpers
./go fetch ohNet
./go fetch openssl
Find all posts by this user
Quote this message in a reply
23-11-2016, 03:14 PM
Post: #3
RE: Http looping on 416
(23-11-2016 01:52 PM)simonc Wrote:  What HTTP headers does your server return? I'm particularly interested in the HTTP version it uses (1.0 or 1.1) and whether it reports a content length, uses chunked transfer encoding or just writes unspecified amounts of data then closes the connection.

If your server does report a length, are that many bytes transferred?

Here is the initial request from the player:
Code:
GET /stream1.mp3?player=00:24:1d:de:68:68 HTTP/1.1
Host: 192.168.4.4:9010
Connection: close
Icy-MetaData: 1
Range: bytes=0-
The response from the server:
Code:
HTTP/1.1 200 OK
Connection: close
Content-Type:  audio/mpeg
Date: Wed, 23 Nov 2016 14:33:39 GMT

(data follows)

After the end of the first request, I receive a second one:
Code:
GET /stream1.mp3?player=00:24:1d:de:68:68 HTTP/1.1
Host: 192.168.4.4:9010
Connection: close
Icy-MetaData: 1
Range: bytes=163760-

I think that I should not receive this at all, and I've tried to answer by 416 for example, which gets the player looping on:
Code:
>ProtocolNetwork::Connect
ProtocolNetwork::Open
<Protocol::Connect
ProtocolHttp::WriteRequest send request
ProtocolHttp::WriteRequest read response
ProtocolHttp::WriteRequest response code 416
ProtocolHttp::ProcessContent 0
ProtocolNetwork::Close
>ProtocolNetwork::Connect
ProtocolNetwork::Open
<Protocol::Connect
ProtocolHttp::WriteRequest send request
ProtocolHttp::WriteRequest read response
ProtocolHttp::WriteRequest response code 416

(etc.)

I checked that I can prevent the player from looping by faking a range response to the second request (4 bogus bytes and a length corresponding to the actual data sent + 4). I tried this with wav data hence the 4 bytes choice. Unfortunately I don't think that I can create a 0-length response ? and I'm afraid that the bogus data could create problems. And it's not really clean of course...

(23-11-2016 07:47 AM)medoc92 Wrote:  I tried to build ohpipipeline from git, with no success.

(23-11-2016 01:52 PM)simonc Wrote:  Have you followed the instructions in the README ? If you had errors during dependency fetching, try fetching dependencies one at a time. E.g.
Code:
./go fetch ohWafHelpers
./go fetch ohNet
./go fetch openssl

I may give it another try. I got through the part which uses mono, and then get a python error about a 'no such file or directory' but it does not say what's missing.

Actually I don't really care about the software player, I'm just using it in hope that it would behave the same as Linn hardware. Do you think that the same problem will occur with actual hardware ?

jf
Find all posts by this user
Quote this message in a reply
23-11-2016, 03:28 PM
Post: #4
RE: Http looping on 416
(23-11-2016 03:14 PM)medoc92 Wrote:  
(23-11-2016 01:52 PM)simonc Wrote:  What HTTP headers does your server return? I'm particularly interested in the HTTP version it uses (1.0 or 1.1) and whether it reports a content length, uses chunked transfer encoding or just writes unspecified amounts of data then closes the connection.

If your server does report a length, are that many bytes transferred?

Here is the initial request from the player:
Code:
GET /stream1.mp3?player=00:24:1d:de:68:68 HTTP/1.1
Host: 192.168.4.4:9010
Connection: close
Icy-MetaData: 1
Range: bytes=0-
The response from the server:
Code:
HTTP/1.1 200 OK
Connection: close
Content-Type:  audio/mpeg
Date: Wed, 23 Nov 2016 14:33:39 GMT

(data follows)

I don't think that's a valid response. Shouldn't a HTTP 1.1 server supply either a Content-Length header or set Transfer-Encoding: chunked? It looks like your server's response is using the old HTTP 1.0 approach of just closing the connection to indicate end of stream; this isn't valid in HTTP 1.1.

That said, from a quick look at the code, I don't think ProtocolHttp will cope properly with HTTP 1.0 servers that don't specify Content-Length. I'll log this as a bug but I'm afraid it won't be a high priority to address.

Any of the following should get your server working with ohPipeline:
  • Include Content-Length in your response.
  • If you don't always know the length in advance, use chunking in your response. When you reach the end of the resource you're serving, just send a 0 length chunk to indicate end-of-stream
  • If neither of the above suit, you could recognise a range request beyond the end of the resource and return a Content-Length: 0 header to indicate no data remains
Find all posts by this user
Quote this message in a reply
23-11-2016, 03:30 PM
Post: #5
RE: Http looping on 416
(23-11-2016 03:14 PM)medoc92 Wrote:  Actually I don't really care about the software player, I'm just using it in hope that it would behave the same as Linn hardware. Do you think that the same problem will occur with actual hardware ?

Yes. Recent Linn firmware releases use ohPipeline so will behave similarly to the Pi player.
Find all posts by this user
Quote this message in a reply
23-11-2016, 03:47 PM
Post: #6
RE: Http looping on 416
(23-11-2016 03:28 PM)simonc Wrote:  I don't think that's a valid response. Shouldn't a HTTP 1.1 server supply either a Content-Length header or set Transfer-Encoding: chunked? It looks like your server's response is using the old HTTP 1.0 approach of just closing the connection to indicate end of stream; this isn't valid in HTTP 1.1.

I think it is valid for an HTTP 1.1 server to do this for a response. (It would not be allowed for a request.) See this section, specifically:

5.By the server closing the connection. (Closing the connection cannot be used to indicate the end of a request body, since that would leave no possibility for the server to send back a response.)
Find all posts by this user
Quote this message in a reply
23-11-2016, 03:52 PM
Post: #7
RE: Http looping on 416
(23-11-2016 03:47 PM)simoncn Wrote:  
(23-11-2016 03:28 PM)simonc Wrote:  I don't think that's a valid response. Shouldn't a HTTP 1.1 server supply either a Content-Length header or set Transfer-Encoding: chunked? It looks like your server's response is using the old HTTP 1.0 approach of just closing the connection to indicate end of stream; this isn't valid in HTTP 1.1.

I think it is valid for an HTTP 1.1 server to do this for a response. (It would not be allowed for a request.) See this section, specifically:

5.By the server closing the connection. (Closing the connection cannot be used to indicate the end of a request body, since that would leave no possibility for the server to send back a response.)

Thanks, you're right.

Unfortunately, this'll still take some time to address. The best bet for a short-term solution will still be any of the options I listed above.
Find all posts by this user
Quote this message in a reply
23-11-2016, 04:13 PM (This post was last modified: 23-11-2016 04:34 PM by medoc92.)
Post: #8
RE: Http looping on 416
(23-11-2016 03:28 PM)simonc Wrote:  I don't think that's a valid response. Shouldn't a HTTP 1.1 server supply either a Content-Length header or set Transfer-Encoding: chunked? It looks like your server's response is using the old HTTP 1.0 approach of just closing the connection to indicate end of stream; this isn't valid in HTTP 1.1.

That said, from a quick look at the code, I don't think ProtocolHttp will cope properly with HTTP 1.0 servers that don't specify Content-Length. I'll log this as a bug but I'm afraid it won't be a high priority to address.

Any of the following should get your server working with ohPipeline:
  • Include Content-Length in your response.
  • If you don't always know the length in advance, use chunking in your response. When you reach the end of the resource you're serving, just send a 0 length chunk to indicate end-of-stream
  • If neither of the above suit, you could recognise a range request beyond the end of the resource and return a Content-Length: 0 header to indicate no data remains

Can't use Content-Length (length not known). Using a chunked encoding would be possible but a bit awkward as the Linn client is the first I see with this issue.

I could try the Content-Length: 0 approach. What response code do you think I should use. 206?

And no Content-Range I guess ?

I tried the Content-Lentgth 0 approach with 200 and 206, and this does not work, the player loops.

Adding bogus bytes works at the http level, but it causes issues with mpeg (works with wav but there will be audible issues).

So the conclusion would be that the only possibility is chunked encoding ?
Find all posts by this user
Quote this message in a reply
24-11-2016, 05:31 PM
Post: #9
RE: Http looping on 416
Kept working on this and here are a few observations:

Playing an mp3 from minimserver (none of my code involved): mp3 gets downloaded 5 times (wireshark capture available if anyone is interested).

Lots of cases where ohplayer loops retrying unretryable errors like 404

Is there any sure way to prevent it from trying to read the same thing several times ? I know that GET is supposedly idempotent, but just not with my data.

And of course, if the server does not indicate that ranges are supported, then ranges are not supported, and it's not a good idea to send a range query to test the eof after a connection close.

I can get my thing to limp along in most cases, but it all looks very brittle.

Actually, I can freeze it very easily just inserting MinimServer tracks from Bubble UPnP.

I just can't believe that it's the same software running in your renderers.
Find all posts by this user
Quote this message in a reply
24-11-2016, 06:21 PM
Post: #10
RE: Http looping on 416
(24-11-2016 05:31 PM)medoc92 Wrote:  And of course, if the server does not indicate that ranges are supported, then ranges are not supported, and it's not a good idea to send a range query to test the eof after a connection close.

What do you mean by "the server does not indicate that ranges are supported"? The absence of the Accept-Ranges header does not necessarily mean that the server does not support ranges (see this section of the HTTP specification).
Find all posts by this user
Quote this message in a reply
Post Reply 


Forum Jump: