Icecast Streaming Media Server Forum Index Icecast Streaming Media Server
Icecast is a Xiph Foundation Project
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Icecast 2.3.1 optimization help for large server needed

 
Post new topic   Reply to topic    Icecast Streaming Media Server Forum Index -> Icecast Server
View previous topic :: View next topic  
Author Message
Anonymous
Guest





PostPosted: Fri Mar 10, 2006 8:28 am    Post subject: Icecast 2.3.1 optimization help for large server needed Reply with quote

I'm running Icecast 2.3.1 for a group of radio stations, and am having trouble with Icecast's status.xsl page seeming to die periodically (it works one minute, and then you get a page not found -- only way to get it back is to restart the icecast process). I'm also experiencing a bandwidth bottleneck somewhere that is limiting the number of simultaneous listeners to the streams, even though the total number of listeners on the server is less than half of the maximums specified in the .xml config file. The system is a dual xeon 2.4GHz system running SuSE 9.2 Professional, connected to a 100Mb pipe to the Internet (and it is a full 100Mb pipe costing me $2000 a month). This server is hosting 12 MP3/MP3pro streams ranging from 16k to 96k in bandwidth, and nearly 2200 simultaneous users (and growing). Average throughput is around 45Mbs, before traffic seems to bottleneck. Hardware has been ruled out, as well as things on my server co-location end. Any suggestions on where to start looking at problems?
Back to top
karlH
Code Warrior
Code Warrior


Joined: 13 Jun 2005
Posts: 5476
Location: UK

PostPosted: Fri Mar 10, 2006 11:27 am    Post subject: Reply with quote

The error log is the place to look for failures or limits being reached. In these error cases what does icecast log.

karl.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Anonymous
Guest





PostPosted: Fri Mar 10, 2006 3:48 pm    Post subject: Reply with quote

Here's a snippit from the error log...

Quote:
[2006-03-02 13:27:16] WARN fserve/fserve_client_create req for file "/web/wayh_high.smi" No such file or directory
[2006-03-02 13:29:59] WARN fserve/fserve_client_create req for file "/web/wayw_high.smi" No such file or directory
[2006-03-02 13:40:39] WARN fserve/fserve_client_create req for file "/web/wayw_high.smi" No such file or directory
[2006-03-02 13:45:30] WARN fserve/fserve_client_create req for file "/web/wayf_high.smi" No such file or directory
[2006-03-02 13:47:18] WARN fserve/fserve_client_create req for file "/web/wayf_high.smi" No such file or directory
[2006-03-02 13:48:34] WARN fserve/fserve_client_create req for file "/web/wayw_high.smi" No such file or directory
[2006-03-02 14:01:00] WARN client/client_read_bytes source connection has died
[2006-03-02 14:01:02] WARN client/client_read_bytes source connection has died
[2006-03-02 14:03:22] WARN fserve/fserve_client_create req for file "/web/wayf_high.smi" No such file or directory
[2006-03-02 14:05:33] WARN client/client_read_bytes source connection has died
[2006-03-02 14:07:04] WARN fserve/fserve_client_create req for file "/web/wayw_high.smi" No such file or directory
[2006-03-02 14:12:07] WARN fserve/fserve_client_create req for file "/web/wayf_high.smi" No such file or directory
[2006-03-02 14:13:55] WARN client/client_read_bytes source connection has died
[2006-03-02 14:14:04] WARN client/client_read_bytes source connection has died
[2006-03-02 14:25:55] WARN fserve/fserve_client_create req for file "/web/wayw_high.smi" No such file or directory
[2006-03-02 14:28:09] WARN fserve/fserve_client_create req for file "/web/wayw_high.smi" No such file or directory
[2006-03-02 14:28:40] WARN fserve/fserve_client_create req for file "/web/wayh_high.smi" No such file or directory
[2006-03-02 14:28:59] WARN fserve/fserve_client_create req for file "/web/wayf_high.smi" No such file or directory
[2006-03-02 14:32:33] WARN fserve/fserve_client_create req for file "/web/wayf_high.smi" No such file or directory
[2006-03-02 14:41:27] WARN fserve/fserve_client_create req for file "/web/wayw_high.smi" No such file or directory
[2006-03-02 14:48:05] WARN fserve/fserve_client_create req for file "/web/wayw_high.smi" No such file or directory
[2006-03-02 14:49:31] WARN fserve/fserve_client_create req for file "/web/wayf_high.smi" No such file or directory
[2006-03-02 14:57:28] WARN fserve/fserve_client_create req for file "/web/wayf_high.smi" No such file or directory
[2006-03-02 15:15:26] WARN fserve/fserve_client_create req for file "/web/kxwa_high.smi" No such file or directory
[2006-03-02 15:40:57] WARN fserve/fserve_client_create req for file "/web/wayh_high.smi" No such file or directory
[2006-03-02 15:51:18] WARN fserve/fserve_client_create req for file "/web/kxwa_high.smi" No such file or directory
[2006-03-02 16:09:28] WARN fserve/fserve_client_create req for file "/web/kxwa_high.smi" No such file or directory
[2006-03-02 16:15:05] WARN fserve/fserve_client_create req for file "/web/kxwa_high.smi" No such file or directory
[2006-03-02 16:28:14] WARN fserve/fserve_client_create req for file "/web/wayw_high.smi" No such file or directory
[2006-03-02 16:31:06] WARN fserve/fserve_client_create req for file "/web/wayf_high.smi" No such file or directory
[2006-03-02 16:32:53] WARN fserve/fserve_client_create req for file "/web/wayf_high.smi" No such file or directory
[2006-03-02 16:37:31] WARN fserve/fserve_client_create req for file "/web/wayw_high.smi" No such file or directory
[2006-03-02 16:52:26] WARN fserve/fserve_client_create req for file "/web/wayw_high.smi" No such file or directory
[2006-03-02 17:03:05] WARN fserve/fserve_client_create req for file "/web/wayf_high.smi" No such file or directory
[2006-03-02 17:03:47] WARN fserve/fserve_client_create req for file "/web/kxwa_high.smi" No such file or directory
[2006-03-02 17:19:31] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:19:31] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:18] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:18] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:19] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:19] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:20] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:20] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:20] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:20] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:21] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:21] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:21] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:21] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:21] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:22] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:22] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:23] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:23] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:23] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:24] WARN connection/_accept_connection accept() failed with error 24: Too many open files
[2006-03-02 17:20:24] WARN connection/_accept_connection accept() failed with error 24: Too many open files

Once the "Too many open files" line appears in the log, Icecast will no longer display any status page, the admin interface dies, and it will start dropping listeners over time. In Icecast.xml, the max listeners parameter is set to 3000, while just before the "crash", there were only 1500-1600 listeners to the various streams. There are no relays and no files being served by icecast's webserver, other than the status page.
Back to top
karlH
Code Warrior
Code Warrior


Joined: 13 Jun 2005
Posts: 5476
Location: UK

PostPosted: Fri Mar 10, 2006 5:13 pm    Post subject: Reply with quote

Do you know what your process file descriptor limit is ? The accept fails because that is being reached and it's not an icecast setting. The setting, if set, will be in /etc/security/limits.conf (nofile)

Can you also try some fixes that have been collected since the 2.3.1 release, I have a tarball at http://mediacast1.com/~karl/testing/icecast-2.3.1a.tar.gz

karl.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Anonymous
Guest





PostPosted: Fri Mar 10, 2006 7:36 pm    Post subject: Reply with quote

I don't know what the file descriptor limit is, unfortunately. The install is an out-of-the-box Suse 9.2 Professional load, with the usual security updates and some lockdowns to keep the script kiddies out. /etc/security/limits.conf is set at its defaults, with all lines commented out. Is there another place I should look?

I'll check out the fixes and see if any help. Thanks for your help on all of this!
Back to top
karlH
Code Warrior
Code Warrior


Joined: 13 Jun 2005
Posts: 5476
Location: UK

PostPosted: Fri Mar 10, 2006 8:22 pm    Post subject: Reply with quote

The limits file states various limits that you want to impose, ulimit -a and ulimit -aH will show what the soft/hard limits currently are but typically the limit has been 1024 descriptors. I guess you will need to increase this limit if you want a lot more listeners to connect.

karl.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Anonymous
Guest





PostPosted: Fri Mar 10, 2006 11:12 pm    Post subject: Reply with quote

Check the file-max entry in proc:

Code:

cat /proc/sys/fs/file-max


And you can raise the system default value from /etc/sysctl.conf

add the following line to it:
Code:

sys.fs.file-max=VALUE

And run the sysctl program to apply:
Code:

sysctl -p


If you run icecast as a non-root user, you have the raise the ulimit. This can't be higher than the system value displayed in proc.
Back to top
Anonymous
Guest





PostPosted: Sat Mar 11, 2006 5:52 pm    Post subject: Reply with quote

/proc/sys/fs/file-max was set to over 200000. Trying to use /etc/security/limits.conf didn't work either at first. I ended up doing a little more googling and found an article about Oracle and the mods needed to make the Oracle dB work on Suse 9.2. Turns out you have to install the ulimits package using Yast2 in order to control anything with the ulimit command, otherwise you will get a "operation not permitted" error. Going to test this over the next week and see if there is any improvement.
Thanks again.
Back to top
Display posts from previous:   
Post new topic   Reply to topic    Icecast Streaming Media Server Forum Index -> Icecast Server All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2002 phpBB Group
subRebel style by ktauber