Icecast Streaming Media Server Forum Index Icecast Streaming Media Server
Icecast is a Xiph Foundation Project
 
 FAQFAQ   SearchSearch   MemberlistMemberlist   UsergroupsUsergroups   RegisterRegister 
 ProfileProfile   Log in to check your private messagesLog in to check your private messages   Log inLog in 

2.3.2 SIGHUP not working correctly?

 
Post new topic   Reply to topic    Icecast Streaming Media Server Forum Index -> Bug Reports
View previous topic :: View next topic  
Author Message
jaba



Joined: 21 Sep 2009
Posts: 8
Location: Helsinki, Finland

PostPosted: Mon Sep 21, 2009 12:40 pm    Post subject: 2.3.2 SIGHUP not working correctly? Reply with quote

Hello,

I'm relatively new with Icecast 2, so this might well be a problem between my keyboard and chair, but nevertheless I've confirmed this bug with both FreeBSD 6.2 and Debian Lenny 5.0.3. I even tried to fetch the latest Icecast build from Debian Sid (which claims to have some logrotate related problems fixed), but no go.

With Icecast 2.3.2 logrotate makes things go wrong. icecast2 will still be hanging around in the process list and currently connected clients will be able to continue listening to radio, but admin interface will show a blank screen and new clients are unable to connect.

This is somehow related to configuration reloading and parsing. My best guess is that even though Icecast thinks it was able to correctly re-read and parse the config file, in reality it was unable to do so. Perhaps some kind of race condition with all the locks Icecast seems to create under the hood?

I have no real knowledge about Icecast source code - today I glimpsed through its code for the first time a bit. But anyway, as soon as I slightly changed the src/slave.c, logrotate started to work as I want.

Code:
diff --git a/src/slave.c b/src/slave.c
index cb792ac..9de9157 100644
--- a/src/slave.c
+++ b/src/slave.c
@@ -695,9 +695,10 @@ static void *_slave_thread(void *arg)
         /* re-read xml file if requested */
         if (global . schedule_config_reread)
         {
-            event_config_read (NULL);
+            /*event_config_read (NULL);*/
+            restart_logging (config);
             global . schedule_config_reread = 0;
-        }
+        }


In my case on-the-fly config modifications are very rare, so I can live with the fact I have to do a full stop and start for every config change. On the other hand, I do need to make Icecast survive during logrotate, so this arguably stupid little patch made my day for now.

Still a proper fix should be made, or if you have any suggestions what could trigger this logrotate crash behaviour, all the hints are gladly accepted.
Back to top
View user's profile Send private message Visit poster's website
karlH
Code Warrior
Code Warrior


Joined: 13 Jun 2005
Posts: 5476
Location: UK

PostPosted: Mon Sep 21, 2009 1:43 pm    Post subject: Reply with quote

more detail on the failure case would be helpful. One possible issue with reload is if changeowner or chroot is enabled because obviously you are changing your access after starting up, although symlinks/permissions can get around that if you are careful.

karl.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
jaba



Joined: 21 Sep 2009
Posts: 8
Location: Helsinki, Finland

PostPosted: Mon Sep 21, 2009 1:48 pm    Post subject: Reply with quote

changeowner and chroot are not in use, Icecast2 will start as icecast2 user thanks to Debian /etc/init.d/icecast2 init script.

I'll provide you a more detailed report tomorrow, I'll need to investigate this a bit further.
Back to top
View user's profile Send private message Visit poster's website
jaba



Joined: 21 Sep 2009
Posts: 8
Location: Helsinki, Finland

PostPosted: Tue Sep 22, 2009 3:32 am    Post subject: Reply with quote

I will soon start my further investigation, but here's couple of other random notes about the things I made before I started hacking the source code:

- To make sure /etc/init.d/icecast2 init script is not to blame, I started icecast2 manually with icecast2 -b -c /etc/icecast2/icecast.xml, and also tried the foregrounded version without the -b option.

- To make sure SMP is not to blame, I started icecast manually with taskset 0x1 icecast2 -b -c /etc/icecast2/icecast.xml, and also without the -b option. On the other hand, my FreeBSD 6.2 server is a very old Pentium III, so surely that's a uni-processor system, anyway.

- Logs did not reveal any odd error messages.

- I tried to strace the icecast2 processes whenever they were stuck, but did not find anything. I'll try again today and paste the results for you to see.
Back to top
View user's profile Send private message Visit poster's website
jaba



Joined: 21 Sep 2009
Posts: 8
Location: Helsinki, Finland

PostPosted: Tue Sep 22, 2009 7:02 am    Post subject: Reply with quote

I still have no idea what is going on, but here's one more finding: after HUP the web admin interface stops working because suddenly the password doesn't match. Icecast thinks that password the client supplied was GET /status.xsl HTTP/1.0 or whatever http request I make.

Code:
2009-09-22  09:49:24 INFO connection/_handle_shoutcast_compatible password does not match "GET /status.xsl HTTP/1.0"
Back to top
View user's profile Send private message Visit poster's website
karlH
Code Warrior
Code Warrior


Joined: 13 Jun 2005
Posts: 5476
Location: UK

PostPosted: Tue Sep 22, 2009 3:08 pm    Post subject: Reply with quote

so icecast believes that is coming in on a shoutcast port. The thing is that 2.3.2 does not reopen the listening sockets over HUP so those should stay the same. I'll check here but I don't have FBSD for comparison.

karl.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
jaba



Joined: 21 Sep 2009
Posts: 8
Location: Helsinki, Finland

PostPosted: Wed Sep 23, 2009 5:53 am    Post subject: Reply with quote

But this happens with Debian Lenny 5.0.3 for me, too. Actually that's the main server where I'm trying to track down this issue. FreeBSD was just checking out if that happens elsewhere or if that's a bug in Debian.
Back to top
View user's profile Send private message Visit poster's website
karlH
Code Warrior
Code Warrior


Joined: 13 Jun 2005
Posts: 5476
Location: UK

PostPosted: Wed Sep 23, 2009 10:11 am    Post subject: Reply with quote

Did you try the current trunk code at people.xiph.org/~brendan/snapshots? I can't try 2.3.2 here as a glibc symbol redefinition is stopping that.

karl.
Back to top
View user's profile Send private message Send e-mail Visit poster's website
jaba



Joined: 21 Sep 2009
Posts: 8
Location: Helsinki, Finland

PostPosted: Wed Sep 23, 2009 10:25 am    Post subject: Reply with quote

No, I didn't try the trunk code yet. Downloading it as we speak.
Back to top
View user's profile Send private message Visit poster's website
jaba



Joined: 21 Sep 2009
Posts: 8
Location: Helsinki, Finland

PostPosted: Wed Sep 23, 2009 10:54 am    Post subject: Reply with quote

Or, to be more accurate, I tried to download the latest trunk tgz. For some reason my wget is not amused at the moment, it just tries to connect to people.xiph.org, but does not begin the download. Will try again later.
Back to top
View user's profile Send private message Visit poster's website
jaba



Joined: 21 Sep 2009
Posts: 8
Location: Helsinki, Finland

PostPosted: Thu Sep 24, 2009 6:33 am    Post subject: Reply with quote

I'm sorry to tell you, but the trunk version does not work any better. The behaviour is exactly the same. New clients are unable to connect and admin page shows just a blank page.

Here's what the log tells immediately after HUP signal:

---
[2009-09-24 09:28:40] DBUG stats/modify_node_event update "/ourradiourl" total_bytes_read (306366)
[2009-09-24 09:28:40] DBUG stats/modify_node_event update "/ourradiourl" total_bytes_sent (242473)
[2009-09-24 09:28:42] DBUG yp/yp_recheck_config Updating YP configuration
[2009-09-24 09:28:42] DBUG stats/modify_node_event update global server_id (Icecast trunk)
[2009-09-24 09:28:42] DBUG stats/modify_node_event update global host (ouricecastserver.ourdomain.com)
[2009-09-24 09:28:42] DBUG stats/modify_node_event update global location (Earth)
[2009-09-24 09:28:42] DBUG stats/modify_node_event update global admin (icemaster@localhost)
[2009-09-24 09:28:43] DBUG slave/_slave_thread checking master stream list
[2009-09-24 09:28:44] INFO connection/_handle_shoutcast_compatible password does not match "GET /ourradiourl HTTP/1.1 Accept: */*"
[2009-09-24 09:28:44] DBUG stats/modify_node_event update global clients (3)
[2009-09-24 09:28:44] DBUG stats/modify_node_event update global connections (62)
[2009-09-24 09:28:44] DBUG stats/modify_node_event update global clients (2)
[2009-09-24 09:28:45] DBUG stats/modify_node_event update "/ourradiourl" total_bytes_read (386204)
[2009-09-24 09:28:45] DBUG stats/modify_node_event update "/ourradiourl" total_bytes_sent (322278)
[2009-09-24 09:28:45] INFO connection/_handle_shoutcast_compatible password does not match "GET /admin/ HTTP/1.1"
[2009-09-24 09:28:45] DBUG stats/modify_node_event update global clients (3)
[2009-09-24 09:28:45] DBUG stats/modify_node_event update global connections (63)
[2009-09-24 09:28:45] DBUG stats/modify_node_event update global clients (2)
[2009-09-24 09:28:45] INFO connection/_handle_shoutcast_compatible password does not match "GET /admin/ HTTP/1.1"
---
Back to top
View user's profile Send private message Visit poster's website
Display posts from previous:   
Post new topic   Reply to topic    Icecast Streaming Media Server Forum Index -> Bug Reports All times are GMT
Page 1 of 1

 
Jump to:  
You cannot post new topics in this forum
You cannot reply to topics in this forum
You cannot edit your posts in this forum
You cannot delete your posts in this forum
You cannot vote in polls in this forum


Powered by phpBB © 2001, 2002 phpBB Group
subRebel style by ktauber