Icecast Streaming Media Server

[gee2803]

Hello,

because of a server change we had to change the limit settings in icecast. Right now the stream does not make problems anymore, if we increase the burst-size to 700320. I calculated this value for 30 seconds in 192 kbit/s, but it seems to be 42 seconds in VLC player.

Anyway: The metadata is wrong now. The new title and artist are displayed before the stream actually brings up this song.

Do we have to change the queue-size, too?

The icecast documentation looked like we just have to change the queue-size, if having network problems, but the queue-size didn't make the stream work properly. It stopped over and over again.

(queue-size is like originally set to 524288)

Thank you in advance!

[karlH]

the queue size is the maximum length of the queue, the burst is a portion of it so if you want 700k burst then make the queue longer.

The fact that it's 42 seconds not 30 is not surprising, it may take you a few seconds to get that 700k burst and then the queue from those few seconds. Note the any network lag or even player scheduling can cause a stall in the playback. You may not want 30 seconds

As for the metadata, when you are dealing with very large bursts then don't be surprised you find these player bugs. What you are seeing is the displaying of the metadata as it is received, but the playback of audio is going through a buffer which is at least 30 seconds or whatever the player allows for.

karl.

[gee2803]

Thank you. So what would you set for a 192 kbit/s stream, which worked perfectly in Germany but has problems with servers without Germany?

My idea was a bigger cache to prevent the player to lose its connection. The standard setting worked good with the old network.

The pings to the new servers are not that much higher. Its like about 5ms, but every 5 minutes the stream stocks for a second and at least once an hour the connection is lost.

I just wanted to increase the buffer a bit, but didn't find out what to change as burst-size was mentioned only to be changed if needed and queue-size for networking problems.

Could you explain how to calculate the best way for cache and burst size? Thank you for your help.

[karlH]

For audio, you do not need a vast amount, maybe a bit more than normal for larger bitrates like 192k, ~150k (4secs*24k) burst should be enough. The default queue (512k) will be about 20 seconds max, so you may want to increase that, eg 1024k.

If the problems you are seeing are down to slow listeners (being too slow after a certain amount of time has elapsed), then it could be the TCP settings are throttling the link for people further away from the server. A fixed windows size may be too small for that bitrate required over that distance. I did include an option in the kh builds for this sort of thing in cases where tcp window scaling was disabled <so-sndbuf>64000</so-sndbuf> in <mount> or <listen-socket>

karl.

[gee2803]

Thank you. Is a config reload enough or do I have to restart icecast?
How can I check whether tcp window scaling is disabled or not? I don't understand this feature, yet.

Right now it is working good with just a lag an hour I think by using

<limits>
<clients>10000</clients>
<sources>500</sources>
<!-- <queue-size>524288</queue-size> -->
<queue-size>1572864</queue-size>
<client-timeout>30</client-timeout>
<header-timeout>15</header-timeout>
<source-timeout>10</source-timeout>
<!-- same as burst-on-connect, but this allows for being more
specific on how much to burst. Most people won't need to
change from the default 64k. Applies to all mountpoints -->
<!-- <burst-size>65535</burst-size> -->
<burst-size>65535</burst-size>
<max-bandwidth>1000M</max-bandwidth>
</limits>

I tried a queue-size of 1048576, but that cause more problems then now.
The difference from studio to listener is just 8 seconds now.

[gee2803]

The setting didn't resolve the problem. And with my new queue size the stream tag is delivered 17 seconds before the song is played. I really need to get those in sync.

What I don't understand is that you wrote the standard queue-size 512k will be 20 seconds. With the 3 times higher queue-size of 1572864 it is just 8 seconds!

So the meta tag is probably at the right point of time, but the stream comes way to early.

What am I doing wrong?

[karlH]

Well you haven't actually said much on the limits you are imposing, Assuming you only have this stream on the server (although you have 500 source limit) and 1Gb which is not being reached, and no limit override on a <mount> then you have specified ~1.5Meg queue size and 64k burst.

First thing to understand, queue-size is how much a listener is allowed to lag before being dropped by icecast, it has nothing to do with metadata updating. burst size is how much of the queue gets sent quickly (I did tunings on the data rates in kh32 which may help at 192k). If burst is really is 64k then at 192k, that is around 2.5-3 seconds burst, you could try 100k to get somewhere near 4 seconds burst.

I would update to kh32 first then set burst size to 100k at least in <mount> and let me know how is reacts. If the TCP windowing needs fixing larger then you can set <so-sndbuf> in mount but usually you do not need that unless you are on windows.

karl.

[gee2803]

Thank you for your patience.
I just downloaded KH32, didn't see github, yet.

At the moment I am running

<limits>
<clients>10000</clients>
<sources>500</sources>
<!-- <queue-size>524288</queue-size> -->
<queue-size>1048576</queue-size>
<client-timeout>30</client-timeout>
<header-timeout>15</header-timeout>
<source-timeout>10</source-timeout>
<!-- same as burst-on-connect, but this allows for being more
specific on how much to burst. Most people won't need to
change from the default 64k. Applies to all mountpoints -->
<!-- <burst-size>65535</burst-size> -->
<burst-size>131070</burst-size>
<max-bandwidth>1000M</max-bandwidth>
</limits>

Here is my scenario to make it more clear:

We are serving 10.000 listeners on 5 channels.
I am running 3 servers with 1000 Mbit connection.
The main server has 5 MP3, 5 Backup-MP3 and 5 AAC-Streams.
192 kbit/s on MP3, 48 kbit/s on AAC. MP3 mountpoints are limited to 1000 listeners, AAC to 100.

Sources is on 500 to just have a high number. Of course there are not so many sources. 10 from studio, 5 backups from ices.

The 2 slave servers have the same limit section but unlimited on each mountpoint to get weighted balance, which works great, after the master has reached 1000 on one mountpoint.

I'll start KH32 now with
<queue-size>524288</queue-size>
<burst-size>102400</burst-size>

[gee2803]

Sorry Karl, but KH32 doesn't work good in my case. The stream is jumping around, it is not possible to listen and it doesn't matter what limits I set.

I switched back to KH31, because this one brings up correct sound.

Looks like you changed a lot in that one, but I haven't found the changelog. In my case the stream garbles and doesn't work at all.

The cache and burst size works in my case like queue-size is the first package the listener gets and burst-size like the position he starts listening.

If I use
<queue-size>1048576</queue-size>
<burst-size>131070</burst-size>
the song is played first before the meta is send.

If I use
<queue-size>1048576</queue-size>
<burst-size>65535</burst-size>
the meta is send before the song plays.

If I use
<queue-size>524288</queue-size>
no matter what burst-size
the stream connection stops and stops.

[karlH]

you'll have to explain what you exactly mean by "the stream connection stops and stops." This is where looking at the log files becomes helpful.

For mp3/aac, metadata is sent at regular intervals, starting at least so many k (default 16k) into the stream (this is part of the spec), but as I've explained before, the display of metadata and playback of audio is very player dependent, eg what is the buffer size of the player and what is the pre-buffer level?

The key thing to find out is __why__ the connection stops but at the moment we can only assume that all queue sizes on both the master and slaves is 1Meg, you haven't confirmed that, are the relays on the slaves on-demand?

karl.

[gee2803]

It behaves like my internet would be too slow to receive the stream. So it stops, because the data cannot be received too fast.

The KH32 was jumping, so that you can not even understand the words, while only every third word is being received.

I am just using VLC with standard configuration and other players like WMP and flash based webplayer. All behave the same and listeners are reporting same problems, too.

The queue and burst settings are set the same on each server.

The error log gives a lot of

[karlH]

the jumping description sounds like you were not getting the required 192k stream. try the following

curl -o /dev/nulll http://..../mountpoint

see if you have trouble maintaining the bandwidth required to keep the stream going. find out what the ping time is to the server

You could try <so-sndbuf>65535</so-sndbuf> in the <mount>

karl.

[gee2803]

curl show up this

with 1048576 queue-size and 131070 burst-size (running without problems)

[karlH]

the average on the problem run is lower than expected, what pattern are you seeing in the bandwidth of that connection. You can email me the level 4 log of the session.

It should spike for a couple of seconds to something high but then drop back down to about 24k. An average of 22k seems out of place. The first run is more like I was expecting, about 26k after about 40 seconds. Trying the same code here with 512k q, 64k burst, no intro, no auth but over localhost (1500 mtu) shows the expected behaviour, I'll need to introduce lag in the feed next.

karl.

[gee2803]

I also got a packet loss of 10% from server to studio, but not from studio to server. Could it be my internet connection then?

We just switched to a new test with the old structure with servers in our country. The stream runs well with standard limit settings, but sometimes it stops for a second and continous (like the other servers with best setting double queue and burst).

I asked my internet provider to check my connection and will get the measurement soon.