Icecast Streaming Media Server Forum Index Icecast Streaming Media Server
Icecast is a Xiph Foundation Project
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Maximum total bandwidth limit?

 
Post new topic   Reply to topic    Icecast Streaming Media Server Forum Index -> Dev Branches
View previous topic :: View next topic  
Author Message
robertut



Joined: 31 Aug 2007
Posts: 156

PostPosted: Fri Feb 26, 2010 12:49 pm    Post subject: Maximum total bandwidth limit? Reply with quote

Is there any way to limit the maximum total bandwidth that Icecast can serve?

Our server has 1Gbit internet access, and serves thousands of listeners peaking about 375 Mbit every day (unicast...). In order to protect the server bandwidth, we made a calculation for all the mounts (different bitrates) and limited the maximum number of listeners per each, to make sure the total used bandwidth doesn't go over 980Mbit - so we can still reach and manage the server. The maximum listener numbers per mounts are not equal - we look at the usual listener peaks per mount, and we approximate the limit proportionally.

However, there are certain occasions (sports events for example) when listener interest grows very high on certain mounts. Those mounts get full, while there would be enough bandwidth to serve. So when such thing happens, we lower the limit on the other mounts, and try to raise it on the popular ones, still watching the max. serve-able bandwidth to not go over 980Mbit. And after the popularity goes back to the usual values, we restore the original state. (thus we need to use the KH version of Icecast Win32, because only that one has "Reload config" on the web interface - allowing to change limits without dropping the listener clients)

It would be a lot easier to just maximize the total bandwidth served by Icecast, and not establish any maximum listener count on the mounts. If the max bandwidth is reached, no further listener connections would be accepted, regardless on which mount comes it.

Is there any chance to have that?
Back to top
View user's profile Send private message
karlH
Code Warrior
Code Warrior


Joined: 13 Jun 2005
Posts: 5476
Location: UK

PostPosted: Fri Feb 26, 2010 2:29 pm    Post subject: Reply with quote

in the kh tree you can set in
<limits>
<max-bandwidth>980M</max-bandwidth>

you will want to set it lower than that just to handle any overheads.

karl.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
robertut



Joined: 31 Aug 2007
Posts: 156

PostPosted: Fri Feb 26, 2010 4:20 pm    Post subject: Reply with quote

karlH wrote:
in the kh tree you can set in
<limits>
<max-bandwidth>980M</max-bandwidth>

you will want to set it lower than that just to handle any overheads.

karl.


How much max do you recommend in the case I described above?
Do you think the strategy I described (max-bandwidth instead max listener per mount) is appropriate?
Back to top
View user's profile Send private message
karlH
Code Warrior
Code Warrior


Joined: 13 Jun 2005
Posts: 5476
Location: UK

PostPosted: Fri Feb 26, 2010 4:56 pm    Post subject: Reply with quote

I haven't really investigated to give any figures, you will have protocol overhead. Obviously things like huge bursts (eg lots of new listeners connecting at the same time) will affect the outgoing average on a temporary basis and you can still impose max-bandwidth or max-listeners on a per-mount basis. It only acts as a check for new listeners not an ongoing limiter for existing listeners.

certainly limiting based on bandwidth as opposed to listener counts does allow for certain amount of flexibility and if each mountpoint is busy at different times then that works to your advantage. I suspect you won't want to get rid of the per-mount limits completely, you may not want to prevent access to other streams just because one becomes popular, but having a high upper per-mount limit is fine.

karl.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
robertut



Joined: 31 Aug 2007
Posts: 156

PostPosted: Fri Feb 26, 2010 5:24 pm    Post subject: Reply with quote

karlH wrote:
It only acts as a check for new listeners not an ongoing limiter for existing listeners.
Does this mean that the served bandwidth can go upper than max-bandwidth?
Back to top
View user's profile Send private message
karlH
Code Warrior
Code Warrior


Joined: 13 Jun 2005
Posts: 5476
Location: UK

PostPosted: Fri Feb 26, 2010 7:00 pm    Post subject: Reply with quote

yes. Assuming simple figures, If you have a limit of say 1000k with a stream that is 128k and are currently streaming to 6 listeners, it's certainly possible to that listener number 7 and 8 can connect because the total bandwidth has not yet reached passed that limit even though normally 8 listeners would of put you over the threshold. There is a latency there as listeners connect before bandwidth is used. If listener 7 connects and gets a burst then that alone would probably be enough to put it over the threshold until that burst has finished.

For the global limit, the average over the last few seconds is taken and as long as the bitrate for the new connection would fit then the listener connection is allowed. For per-mount bandwidth limits, the number of listeners +1, multiplied by the bitrate has to be less than the per-mount max bandwidth.

The per-mount streams limit can also be exceeded for bursting new connections as the calculation is primarily based on listener count * incoming bitrate but as the bitrate and counts are know then we are not subject to the latency aspect of the global bandwidth.

Whether something extra can be done to reduce overall bandwidth if the global limit is exceeded is another question. A listener can have delayed scheduling allowing for certain lagging listeners to pushed off the queue, this would bring the global bandwidth down.

karl.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
robertut



Joined: 31 Aug 2007
Posts: 156

PostPosted: Fri Feb 26, 2010 7:57 pm    Post subject: Reply with quote

I see.

Do you think that <max-bandwidth>900M</max-bandwidth> would be safe enough, considering the above and the usual TCP/IP overhead?

The server only serves Icecast, and is a Win2003 machine.
Back to top
View user's profile Send private message
karlH
Code Warrior
Code Warrior


Joined: 13 Jun 2005
Posts: 5476
Location: UK

PostPosted: Fri Feb 26, 2010 8:57 pm    Post subject: Reply with quote

I can't see that being a problem. Obviously if you are doing a 1Meg burst size for each listener then a few of those is going to bounce the bandwidth meter a bit, but if you are talking of 64kbytes (default) then it shouldn't be an issue.

I think I have a good idea on what to do for throttling in cases of excessive bandwidth usage, to allow listeners to lag behind.

karl.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
robertut



Joined: 31 Aug 2007
Posts: 156

PostPosted: Sun Feb 28, 2010 10:46 am    Post subject: Reply with quote

karlH wrote:
I think I have a good idea on what to do for throttling in cases of excessive bandwidth usage, to allow listeners to lag behind.
Wow, can't wait to see it.
Back to top
View user's profile Send private message
Display posts from previous:   
Post new topic   Reply to topic    Icecast Streaming Media Server Forum Index -> Dev Branches All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2002 phpBB Group
subRebel style by ktauber