View previous topic :: View next topic |
Author |
Message |
Anonymous Guest
|
Posted: Wed Sep 14, 2005 1:50 pm Post subject: traffic increase since upgrade from 1.3.x to 2.2 |
|
|
hi folks
we have recently updated our icecast server from 1.3.x to release 2.2 and experience a dramatic increase in the bandwidth traffic allthough listeners and streambandwidth is identical.
we have 300 listeners on a 32kbps and according to the MRTG the traffic is now increased by more than 50% to 16800kbps or 2100kbyte/s on outgoing and 4 times on incoming and even worse, the CPU usage of our border and core routers has more than doubled. in average this points to a streambandwidth of approx. 56kbps.
first we thought it could be related to the title tags, as we did not have them before, but we disabled them and saw no significent change.
we can also see another phaenomenon, that proxy servers, which connect to our stream get far more single bandwidth than a single listener...
single user currently has about 42kpbs, proxies up to 120kbps...
we suspect it is related to the headers that the streambandwidth is no longer 35 kbps for a single stream but now about 42 kbps.
also the packetsize or frequency must be different, else the router cpu would not have increased so dramatically...
has anybody else experiences this?
has anybody got ideas?
has the packetsize changed ?
is there a different way of caching deployed?
and hint is helpfull...
regards
tom |
|
Back to top |
|
|
karlH Code Warrior
Joined: 13 Jun 2005 Posts: 5476 Location: UK
|
Posted: Thu Sep 15, 2005 4:20 pm Post subject: |
|
|
can you try out rc3 and let us know how that works for you in this regard. The passthrough handler (used for mp3 aac etc) now uses packs the data to prevent really small writes to the client. What you may be experiencing is TCP overhead.
karl. |
|
Back to top |
|
|
Anonymous Guest
|
Posted: Fri Sep 16, 2005 7:40 am Post subject: 2.3 rc3 when? |
|
|
hi karl
we will try it out, the minute rc3 is out.
any timetable when the release is out?
tom |
|
Back to top |
|
|
Anonymous Guest
|
Posted: Fri Sep 16, 2005 8:11 am Post subject: transparent proxy option |
|
|
hi karl and the developement team
we suspect that the problem could be related to proxy servers.
they are not treated as a single listener by icecast, but seem to get unlimited bandwidth each time a new client is going trough this proxy server...
in the old versions, there was an option called "transparent proxy" ..that obviously avoided this effect
would it be possible to implement this feature again in RC3?
tom |
|
Back to top |
|
|
karlH Code Warrior
Joined: 13 Jun 2005 Posts: 5476 Location: UK
|
|
Back to top |
|
|
karlH Code Warrior
Joined: 13 Jun 2005 Posts: 5476 Location: UK
|
Posted: Fri Sep 16, 2005 1:35 pm Post subject: Re: transparent proxy option |
|
|
thesmile wrote: |
hi karl and the developement team
we suspect that the problem could be related to proxy servers.
they are not treated as a single listener by icecast, but seem to get unlimited bandwidth each time a new client is going trough this proxy server...
in the old versions, there was an option called "transparent proxy" ..that obviously avoided this effect
would it be possible to implement this feature again in RC3?
tom |
proxy servers, like icecast, should not treat the listeners as the same, in the case of mp3, most want shoutcast metadata others may not. In the case of Ogg, the headers have to come first.
karl. |
|
Back to top |
|
|
Anonymous Guest
|
Posted: Mon Sep 19, 2005 7:18 am Post subject: missunderstanding |
|
|
hi karl
seems to be a missunderstanding.
what I meant was:
we have several listener, who are coming from large companies with a proxy gateway and when you check their IP traffic or bandwidth, they are getting more than 120 kbps and more..
so each time a client within their network is trying to listen to the stream, and connectsthrough their proxy server, he is treated by the icecast server as a new listener, allthough I would expect, that the proxy server is doing this internally.
basically it means, that whenever a proxy server is the client to connect to the icecast server, he is getting unlimited bandwidth and undermines the 300 listener limit!!!!
we will however install 2.3 rc3 today to test it..
regards
tom |
|
Back to top |
|
|
Anonymous Guest
|
Posted: Tue Sep 20, 2005 9:03 am Post subject: 2.3 rc3 solved the problem |
|
|
hi folks
just a short feeback
we installed 2.3rc3 and it seems as if this was the solve for the problem.
traffic-bandwidth and cpu on routers is back to where it was before
I wonder what the reason was...headers??
tom
Last edited by Anonymous on Tue Sep 20, 2005 1:09 pm; edited 1 time in total |
|
Back to top |
|
|
karlH Code Warrior
Joined: 13 Jun 2005 Posts: 5476 Location: UK
|
Posted: Tue Sep 20, 2005 12:56 pm Post subject: |
|
|
Headers only occur once, another possibility could be mp3 metadata as that is repeated and some services issue the same metadata in full even if it doesn't change. We do try to filter for those cases.
The problem here I suspect is just TCP overhead. The source client will be sending out various sized packets and the icecast passthrough mechanism was sending out packets the same size as they were coming in. For a single incoming stream the TCP overhead, if small packets are made (eg 40 bytes), will be fairly small, but if you then multiple that up by say 1000 listeners then you get a lot of wastage.
What 2.3 does is to batch up the reads from the incoming stream into larger blocks eg 1k to make sure the TCP overhead is reduced. Enabling nagle wasn't much use on its own as in many cases the packets were sent off before we got chance to batch up the requests in the TCP stack.
karl. |
|
Back to top |
|
|
|