Icecast Streaming Media Server

[shongale]

I have about 6 stations running 10 streams and only one is giving me a slow listener count. No one else has slow listeners except one and they have 29. They are all set up exactly the same with ice-samplerate=44100;ice-bitrate=64;ice-channels=2
What causes Slow Listeners?
Thanks in advance!

[karlH]

The count increases when a listener drops off the end of the queue. So they are at least <queue-size> bytes behind. The count never decreases.

It may indicate that the queue is too small but a 64k stream is not large, for a default 512k queue, that's over 60 seconds lag allowed. It's more likely to indicate a network outage most likely at the receivers end.

karl.

[shongale]

I assUme by receiver you mean the end listener. So how do I fix?

Here is my Limits Section Hopefully it helps.

<limits>
<clients>5000</clients>
<sources>500</sources>
<threadpool>5</threadpool>
<queue-size>1524288</queue-size>
<client-timeout>30</client-timeout>
<header-timeout>15</header-timeout>
<source-timeout>10</source-timeout>

<!-- DO I NEED TO DISABLE THIS?? -->
<!-- If enabled, this will provide a burst of data when a client
first connects, thereby significantly reducing the startup
time for listeners that do substantial buffering. However,
it also significantly increases latency between the source
client and listening client. For low-latency setups, you
might want to disable this. -->

<burst-on-connect>1</burst-on-connect>
<!-- same as burst-on-connect, but this allows for being more
specific on how much to burst. Most people won't need to
change from the default 64k. Applies to all mountpoints -->
<burst-size>65535</burst-size>
</limits>

[karlH]

yes, receiver being the end listener. hmm, with a queue-size of 1.5Meg you should find plenty of flexibility. The burst on connect is ignored.

If it's the same connection that is always dying then you should investigate it more. A throttling link would show as a client which increases lag so check the listclients admin page or the netstat output for the send buffer usage. I'm sure the error log reports the IP of a dropped listener so it may be be easier to filter on that.

I would find it hard to believe that the default tcp settings would be throttling a 64k stream but obviously if there are errors triggering re-transmissions and timeouts then that will effect performance. The only server issue that I've seen showing like this is mbuf limitations on BSD

karl.

[shongale]

karlH: Once again thanks for your help.
A small sample of my access.log file
Where can I find the header info or the field info?
What does the code after the 200 mean?

69.247.177.72 - - [12/Sep/2011:07:22:11 -0700] "GET /admin/metadata HTTP/1.0" 200 177 "-" "(Mozilla Compatible)" 0
193.218.105.163 - - [12/Sep/2011:07:22:31 -0700] "GET /RetroRadioCalifornia.net HTTP/1.0" 200 13162 "-" "-" 1
193.218.105.163 - - [12/Sep/2011:07:22:52 -0700] "GET /TakilmaFM HTTP/1.0" 200 13191 "-" "-" 1
193.218.105.163 - - [12/Sep/2011:07:22:58 -0700] "GET /tkFMslow HTTP/1.0" 200 13272 "-" "-" 1
69.247.177.72 - - [12/Sep/2011:07:23:01 -0700] "GET /admin/metadata HTTP/1.0" 200 177 "-" "(Mozilla Compatible)" 0
193.218.105.163 - - [12/Sep/2011:07:23:12 -0700] "GET /takilmafm.ogg HTTP/1.0" 200 11398 "-" "-" 1
60.241.81.131 - - [12/Sep/2011:07:23:17 -0700] "GET /RetroRadioCalifornia.net HTTP/1.1" 200 563348 "-" "TuneIn Radio" 255
69.247.177.72 - - [12/Sep/2011:07:24:02 -0700] "GET /admin/metadata HTTP/1.0" 200 177 "-" "(Mozilla Compatible)" 0
69.247.177.72 - - [12/Sep/2011:07:28:07 -0700] "GET /admin/metadata HTTP/1.0" 200 177 "-" "(Mozilla Compatible)" 0

Once again thanks for your help!

[karlH]

The access log won't tell you anything significant (it's basically CLF) with this as it only reports the amount sent (after the 200) and the duration (last). The error log can say more information about what's happening with the server itself. For this though it will only say who is being dropped not why, because it's impossible for icecast to know what is wrong if there is no response.

karl.

[shongale]

I have still not resolved the problem. All I can do is reset the stream mountpoint.
Here is my error.log file since yesterday. I see nothing that relates to slow listeners. But I don't know what most of it means.

[2011-09-12 09:34:15] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/Radio_Memphis" No such file or directory
[2011-09-12 09:34:19] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/Radio_Memphis" No such file or directory
[2011-09-12 09:34:22] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/Radio_Memphis" No such file or directory
[2011-09-12 10:39:33] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/Downtown_Radio" No such file or directory
[2011-09-12 12:27:40] WARN xslt/xslt_get_stylesheet Error checking for stylesheet file "/usr/local/share/icecast/admin/xspf.xsl": No such file or directory
[2011-09-12 12:27:40] EROR xslt/xslt_transform problem reading stylesheet "/usr/local/share/icecast/admin/xspf.xsl"
[2011-09-12 12:41:09] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/robots.txt" No such file or directory
[2011-09-12 13:28:08] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/robots.txt" No such file or directory
[2011-09-12 15:03:54] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/FreeWillRadio.org" No such file or directory
[2011-09-12 15:04:36] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/robots.txt" No such file or directory
[2011-09-12 16:12:03] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/FreeWillRadio.org" No such file or directory
[2011-09-12 19:39:32] WARN source/get_next_buffer Disconnecting source due to socket timeout
[2011-09-12 19:39:34] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/Radio_Memphis" No such file or directory
[2011-09-12 19:55:58] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/robots.txt" No such file or directory
[2011-09-12 19:55:58] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/alternate" No such file or directory
[2011-09-12 20:37:51] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/Downtown_Radio" No such file or directory
[2011-09-12 20:38:22] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/Downtown_Radio" No such file or directory
[2011-09-12 23:36:04] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/robots.txt" No such file or directory
[2011-09-13 00:13:03] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/robots.txt" No such file or directory
[2011-09-13 01:12:24] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/3-Gen" No such file or directory
[2011-09-13 01:20:44] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/robots.txt" No such file or directory
[2011-09-13 03:26:39] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/robots.txt" No such file or directory
[2011-09-13 04:13:42] WARN connection/source_startup Mountpoint /Radio_Memphis in use
[2011-09-13 04:13:48] WARN source/get_next_buffer Disconnecting source due to socket timeout
[2011-09-13 06:45:29] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/robots.txt" No such file or directory
[2011-09-13 06:51:32] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/robots.txt" No such file or directory
[2011-09-13 07:45:02] WARN fserve/fserve_client_create req for file "/usr/local/share/icecast/web/robots.txt" No such file or directory

Once again thanks.

[karlH]

log level is too low, the increase in the count is not strictly speaking a warning, you will need at least level 3 to get the info messages.

karl.

[shongale]

Thanks karlH. I changed it to 3 (it was 2) and I will monitor to see what's happening. Be in touch.