Icecast Streaming Media Server Forum Index Icecast Streaming Media Server
Icecast is a Xiph Foundation Project
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

traffic increase since upgrade from 1.3.x to 2.2

 
Post new topic   Reply to topic    Icecast Streaming Media Server Forum Index -> Icecast Server
View previous topic :: View next topic  
Author Message
Anonymous
Guest





PostPosted: Wed Sep 14, 2005 1:50 pm    Post subject: traffic increase since upgrade from 1.3.x to 2.2 Reply with quote

hi folks

we have recently updated our icecast server from 1.3.x to release 2.2 and experience a dramatic increase in the bandwidth traffic allthough listeners and streambandwidth is identical.

we have 300 listeners on a 32kbps and according to the MRTG the traffic is now increased by more than 50% to 16800kbps or 2100kbyte/s on outgoing and 4 times on incoming and even worse, the CPU usage of our border and core routers has more than doubled. in average this points to a streambandwidth of approx. 56kbps.

first we thought it could be related to the title tags, as we did not have them before, but we disabled them and saw no significent change.

we can also see another phaenomenon, that proxy servers, which connect to our stream get far more single bandwidth than a single listener...
single user currently has about 42kpbs, proxies up to 120kbps...

we suspect it is related to the headers that the streambandwidth is no longer 35 kbps for a single stream but now about 42 kbps.

also the packetsize or frequency must be different, else the router cpu would not have increased so dramatically...

has anybody else experiences this?
has anybody got ideas?
has the packetsize changed ?
is there a different way of caching deployed?
and hint is helpfull... Smile

regards

tom
Back to top
karlH
Code Warrior
Code Warrior


Joined: 13 Jun 2005
Posts: 5476
Location: UK

PostPosted: Thu Sep 15, 2005 4:20 pm    Post subject: Reply with quote

can you try out rc3 and let us know how that works for you in this regard. The passthrough handler (used for mp3 aac etc) now uses packs the data to prevent really small writes to the client. What you may be experiencing is TCP overhead.

karl.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Anonymous
Guest





PostPosted: Fri Sep 16, 2005 7:40 am    Post subject: 2.3 rc3 when? Reply with quote

hi karl
we will try it out, the minute rc3 is out.
any timetable when the release is out?

tom
Back to top
Anonymous
Guest





PostPosted: Fri Sep 16, 2005 8:11 am    Post subject: transparent proxy option Reply with quote

hi karl and the developement team

we suspect that the problem could be related to proxy servers.
they are not treated as a single listener by icecast, but seem to get unlimited bandwidth each time a new client is going trough this proxy server...
in the old versions, there was an option called "transparent proxy" ..that obviously avoided this effect
would it be possible to implement this feature again in RC3?

tom
Back to top
karlH
Code Warrior
Code Warrior


Joined: 13 Jun 2005
Posts: 5476
Location: UK

PostPosted: Fri Sep 16, 2005 1:27 pm    Post subject: Re: 2.3 rc3 when? Reply with quote

thesmile wrote:
hi karl
we will try it out, the minute rc3 is out.
any timetable when the release is out?

tom


actually rc3 is already out http://downloads.xiph.org/releases/icecast/icecast-2.3.0.rc3.tar.gz although I see no announcement has been made yet

karl.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
karlH
Code Warrior
Code Warrior


Joined: 13 Jun 2005
Posts: 5476
Location: UK

PostPosted: Fri Sep 16, 2005 1:35 pm    Post subject: Re: transparent proxy option Reply with quote

thesmile wrote:
hi karl and the developement team

we suspect that the problem could be related to proxy servers.
they are not treated as a single listener by icecast, but seem to get unlimited bandwidth each time a new client is going trough this proxy server...
in the old versions, there was an option called "transparent proxy" ..that obviously avoided this effect
would it be possible to implement this feature again in RC3?

tom


proxy servers, like icecast, should not treat the listeners as the same, in the case of mp3, most want shoutcast metadata others may not. In the case of Ogg, the headers have to come first.

karl.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Anonymous
Guest





PostPosted: Mon Sep 19, 2005 7:18 am    Post subject: missunderstanding Reply with quote

hi karl

seems to be a missunderstanding.

what I meant was:

we have several listener, who are coming from large companies with a proxy gateway and when you check their IP traffic or bandwidth, they are getting more than 120 kbps and more..
so each time a client within their network is trying to listen to the stream, and connectsthrough their proxy server, he is treated by the icecast server as a new listener, allthough I would expect, that the proxy server is doing this internally.
basically it means, that whenever a proxy server is the client to connect to the icecast server, he is getting unlimited bandwidth and undermines the 300 listener limit!!!!

we will however install 2.3 rc3 today to test it..

regards

tom
Back to top
Anonymous
Guest





PostPosted: Tue Sep 20, 2005 9:03 am    Post subject: 2.3 rc3 solved the problem Reply with quote

hi folks

just a short feeback
we installed 2.3rc3 and it seems as if this was the solve for the problem.Very Happy
traffic-bandwidth and cpu on routers is back to where it was before
I wonder what the reason was...headers??

tom


Last edited by Anonymous on Tue Sep 20, 2005 1:09 pm; edited 1 time in total
Back to top
karlH
Code Warrior
Code Warrior


Joined: 13 Jun 2005
Posts: 5476
Location: UK

PostPosted: Tue Sep 20, 2005 12:56 pm    Post subject: Reply with quote

Headers only occur once, another possibility could be mp3 metadata as that is repeated and some services issue the same metadata in full even if it doesn't change. We do try to filter for those cases.

The problem here I suspect is just TCP overhead. The source client will be sending out various sized packets and the icecast passthrough mechanism was sending out packets the same size as they were coming in. For a single incoming stream the TCP overhead, if small packets are made (eg 40 bytes), will be fairly small, but if you then multiple that up by say 1000 listeners then you get a lot of wastage.

What 2.3 does is to batch up the reads from the incoming stream into larger blocks eg 1k to make sure the TCP overhead is reduced. Enabling nagle wasn't much use on its own as in many cases the packets were sent off before we got chance to batch up the requests in the TCP stack.

karl.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    Icecast Streaming Media Server Forum Index -> Icecast Server All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2002 phpBB Group
subRebel style by ktauber