Icecast Streaming Media Server Forum Index Icecast Streaming Media Server
Icecast is a Xiph Foundation Project
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

Running multiple streams of ~320kbit with few listeners each

 
Post new topic   Reply to topic    Icecast Streaming Media Server Forum Index -> Icecast Server
View previous topic :: View next topic  
Author Message
shards



Joined: 07 May 2014
Posts: 3

PostPosted: Wed May 07, 2014 1:13 pm    Post subject: Running multiple streams of ~320kbit with few listeners each Reply with quote

We're a few thinking about creating some personalized streams, customizable via a webpage which then causes a liquidsoap script to run that outputs to icecast.

This means that there will be alot of streams with maybe only 1 listener each.

I have created a small proof of concept on a linux box my local network but i seem to run into alot of buffering after creating around 20-25 streams running side by side each defined by their own liquidsoap script and then outputted with a unique stream name to port 8000 on icecast, the streams are fairly simple with a 5 song m3u of 320 kbit mp3s with a small spot occasionally playing atop the music with a fadein/out.

My config on the test environment is set to allow 200 sources, 200 clients and 200 threads in threadpool.

Is it feasible to run that many icecast streams side by side (and just a matter of hardware?)

Is it a bad idea overall and should i rather have few streams with many listeners and avoid the customization options that would spawn many concurrent streams?

Any info or similar projects to go further from here perhaps?

(can provide sample script if needed)
Back to top
View user's profile Send private message
karlH
Code Warrior
Code Warrior


Joined: 13 Jun 2005
Posts: 5476
Location: UK

PostPosted: Wed May 07, 2014 11:25 pm    Post subject: Reply with quote

The main trunk releases can be poor on this as each stream takes one thread and therefore many streams leads to heavy switching, low numbers of threads won't be a problem but 200+ can get into problems. The threadpool setting will not make any difference as it's deprecated.

The KH tree allows for better scaling with this sort of load, as you can specify a <workers> setting in the <limits> to fix the number of threads to process clients, this way you can have thousands of sources (yes that does exist). Note that clients includes listeners, sources and file requests.

karl.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
shards



Joined: 07 May 2014
Posts: 3

PostPosted: Fri May 09, 2014 9:41 am    Post subject: Reply with quote

Thanks for your reply karlH.

I will try the KH edition.

On the regular version I seem to run into buffering already at around 30 streams on my virtual box with "4" cpus, however i guess it's not nescessarily icecast causing this as im using liquidsoap to output to icecast?

I'm not entirely sure where liquidsoap's responsibility ends and icecast2's starts since i'm kinda new to this area, however im using liquidsoap to output to it, so my performance could also be caused by whatever processing liquidsoap does. Any other software i should be looking at for attaching sources to icecast (or can i do it without and still be able to perform somewhat of the same in terms of spots/volume control etc.)?

Ideally i'd be able to run atleast 100 streams on such a setup, since it otherwise starts getting expensive hosting wise for a box with more cpus.
Back to top
View user's profile Send private message
karlH
Code Warrior
Code Warrior


Joined: 13 Jun 2005
Posts: 5476
Location: UK

PostPosted: Fri May 09, 2014 10:26 am    Post subject: Reply with quote

a couple of points to note. virtual box, while useful, is a virtual machine so does have an impact on the scheduling of tasks, liquidsoap deals with the encoding of the stream (which can include any processing of the audio before encoding) so can be quite cpu intensive if you have many of them (the cpu load should help determine in that regard). liquidsoap, like any source client, sends the encoded stream to icecast via the TCP link so it is not required to be on the same machine as icecast so can help in spreading the encoding load if required.

karl.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
shards



Joined: 07 May 2014
Posts: 3

PostPosted: Fri May 09, 2014 10:45 am    Post subject: Reply with quote

Ok yeah, any source clients similar to liquidsoap that may be a bit lighter on the cpu (or is it negligible since they have to perform more or less the same task)?
Back to top
View user's profile Send private message
karlH
Code Warrior
Code Warrior


Joined: 13 Jun 2005
Posts: 5476
Location: UK

PostPosted: Fri May 09, 2014 12:10 pm    Post subject: Reply with quote

encoding will be the same, no matter what the source client (not sure if there is much difference with different encoder libraries these days), but if there is other work being done like resampling then that will add to it.

karl.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    Icecast Streaming Media Server Forum Index -> Icecast Server All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2002 phpBB Group
subRebel style by ktauber