Forums

Leon
Leon
Offline
Resolved
2 votes
My system seems to have updated this morning from 02:10 - Yum Log
Since than i am getting a HIGH volume of log files - messages - Started Session c4476 of user root - about 36 entries every second??

Apr 6 17:09:48 ZodiacTech systemd: Started Session c789 of user root.
Apr 6 17:09:48 ZodiacTech systemd: Started Session c790 of user root.
Apr 6 17:09:48 ZodiacTech systemd: Started Session c791 of user root.
Apr 6 17:09:48 ZodiacTech systemd: Started Session c792 of user root.
Apr 6 17:09:48 ZodiacTech systemd: Started Session c793 of user root.
Apr 6 17:09:48 ZodiacTech systemd: Started Session c794 of user root.
Apr 6 17:09:48 ZodiacTech systemd: Started Session c795 of user root.
Apr 6 17:09:48 ZodiacTech systemd: Started Session c796 of user root.
Apr 6 17:09:48 ZodiacTech systemd: Started Session c797 of user root.
Apr 6 17:09:48 ZodiacTech systemd: Started Session c798 of user root.
Apr 6 17:09:48 ZodiacTech systemd: Started Session c799 of user root.
Apr 6 17:09:48 ZodiacTech systemd: Started Session c800 of user root.
Apr 6 17:09:48 ZodiacTech systemd: Started Session c801 of user root.
Apr 6 17:09:48 ZodiacTech systemd: Started Session c802 of user root.
Apr 6 17:09:48 ZodiacTech systemd: Started Session c803 of user root.
Apr 6 17:09:48 ZodiacTech systemd: Started Session c804 of user root

Did a "HARD" reset so that i can get in and copy from /var/log/messages

It seems that my system is getting overwhelmed, so bad that i can't login even via ssh anymore.

What i did see before i could not get in anymore was a higher memory usage by clamd

Any idea where i can look to find out what is causing this?
Saturday, April 06 2019, 03:14 PM
Share this post:
Responses (72)
  • Accepted Answer

    Saturday, April 06 2019, 04:14 PM - #Permalink
    Resolved
    0 votes
    This is not going to help too much, but you can get rid of the messages in two ways. On my server I changed the LogLevel. I can't remember what you have to then restart to have the change work.

    It also only masks the issue. I would go for the LogLevel solution as it stops the messages from being created in the first place. The rsyslog solution just junks the messages once they have been created.

    I've also recently had a problem with very high system loads with amavis flat-lining at 100% (i.e 1 core flat out) and killing amavisd did not help as it just restarted. I can't get to the vm I had that on for teh moment, but I think the solution in the end appeared to be a memory limitation. I'd been testing too much on a small VM and not removed things like the proxy/content filter, gateway management and so on. Once I pared things back I was OK, but I suspect this won't help you.
    The reply is currently minimized Show
  • Accepted Answer

    Saturday, April 06 2019, 04:18 PM - #Permalink
    Resolved
    0 votes
    It looks like you can change the loglevel for the current session with:
    systemd-analyze set-log-level notice
    Reference

    The when you reboot the permanent change will take effect
    The reply is currently minimized Show
  • Accepted Answer

    Greg
    Greg
    Offline
    Saturday, April 06 2019, 10:03 PM - #Permalink
    Resolved
    0 votes
    I'm facing a similar situation. Starting to day, I'm getting thousands of notification on two serves. Looks like there were some automatic updates today.

    User root logged in via sudo 2019-04-06 14:54:05
    User root logged out via sudo 2019-04-06 14:54:05
    User root logged in via sudo 2019-04-06 14:54:04
    User root logged out via sudo 2019-04-06 14:54:04
    User root logged out via sudo 2019-04-06 14:54:03
    User root logged in via sudo 2019-04-06 14:54:03
    User root logged out via sudo 2019-04-06 14:54:03
    User root logged in via sudo 2019-04-06 14:54:03
    The reply is currently minimized Show
  • Accepted Answer

    Greg
    Greg
    Offline
    Saturday, April 06 2019, 10:21 PM - #Permalink
    Resolved
    0 votes
    I am in a similar situation. Starting today I am getting thousands of SUDO log in and SUDO log out notifications on two servers. There were some automatic updates this morning.
    The reply is currently minimized Show
  • Accepted Answer

    Sunday, April 07 2019, 07:41 AM - #Permalink
    Resolved
    0 votes
    @Greg,
    If you can get to the command line, please can you do a:
    systemd-analyze set-log-level notice
    Then we can get a chance to see toe wood from the trees. Does it make your command line become more responsive?

    Please then have a look at "top" and see if anything is consuming a lot of resources and that your memory usage is OK
    The reply is currently minimized Show
  • Accepted Answer

    Leon
    Leon
    Offline
    Sunday, April 07 2019, 07:57 AM - #Permalink
    Resolved
    0 votes
    Hi Nick

    I tried "systemd-analyze set-log-level notice" last night, did a reboot and it made no difference.
    I am now not able to access my unit at all
    The reply is currently minimized Show
  • Accepted Answer

    Sunday, April 07 2019, 11:49 AM - #Permalink
    Resolved
    0 votes
    Doing:
    systemd-analyze set-log-level notice
    is only temporary. You lose it on next reboot. To make it permanent you have to edit the LogLevel in /etc/systemd/system.conf.

    Please hang fire with a clean install. I'll try to ping Dave.

    Can you boot into Rescue mode. You may be able to make the edit to /etc/systemd/system.conf from there. Also, on the off-chance, try deleting /var/lib/csplugin-events/events.db.

    If you can, have a look at the logs using "cat", "tail", "more" or "less" and look for anything obvious. If /var/log/messages is huge try:
    sed -i '/Started Session.*of user root/d' /var/log/messages
    note that if you do this on a running version of ClearOS you will need to restart rsyslog or you may find some logging stops.
    The reply is currently minimized Show
  • Accepted Answer

    Sunday, April 07 2019, 04:34 PM - #Permalink
    Resolved
    0 votes
    In 4h30m of uptime I have 735 messages like that. That is easily bearable. I wonder what is going on?
    The reply is currently minimized Show
  • Accepted Answer

    Sunday, April 07 2019, 04:38 PM - #Permalink
    Resolved
    0 votes
    @Leon and Greg,
    Clearcenter are happy to log on and have a look, if you can get to your boxes. Please raise a General Enquiry ticket at https://secure.clearcenter.com/portal/ticket_manage.jsp and submit remote login credentials to https://secure.clearcenter.com/portal/system_password.jsp. Clearly you need to be able to get in to your box to open the incoming SSH port, so I realise it may be difficult.
    The reply is currently minimized Show
  • Accepted Answer

    Greg
    Greg
    Offline
    Sunday, April 07 2019, 09:36 PM - #Permalink
    Resolved
    0 votes
    Nick Howitt wrote:

    @Leon and Greg,
    Clearcenter are happy to log on and have a look, if you can get to your boxes. Please raise a General Enquiry ticket at https://secure.clearcenter.com/portal/ticket_manage.jsp and submit remote login credentials to https://secure.clearcenter.com/portal/system_password.jsp. Clearly you need to be able to get in to your box to open the incoming SSH port, so I realise it may be difficult.


    @Nick,
    Thanks. I remotely shut both servers down as they were becoming unresponsive. I will be local to the boxes Monday morning.
    The reply is currently minimized Show
  • Accepted Answer

    Sunday, April 07 2019, 10:53 PM - #Permalink
    Resolved
    0 votes
    The proper way to turn off notices is to leave the file /etc/systemd/system.conf alone and make a file in /etc/systemd/system.conf.d/clearos-base.conf with the contents:

    [Manager]
    LogLevel=notice


    Since the information is not typically necessary for any actionable items under ClearOS. I will be trying to place this file in the location above as it is normal to have these messages only raise if they meet the level of notice and not information.
    The reply is currently minimized Show
  • Accepted Answer

    Sunday, April 07 2019, 11:36 PM - #Permalink
    Resolved
    0 votes
    I've created this as a bug since info logging is not important from systemd but notice logging is.

    https://gitlab.com/clearos/clearfoundation/system-base/commit/dc613d20105c34390569006079b0803cf1b230b1
    The reply is currently minimized Show
  • Accepted Answer

    Greg
    Greg
    Offline
    Monday, April 08 2019, 04:12 PM - #Permalink
    Resolved
    0 votes
    Nick Howitt wrote:

    @Greg,
    If you can get to the command line, please can you do a:
    systemd-analyze set-log-level notice
    Then we can get a chance to see toe wood from the trees. Does it make your command line become more responsive?

    Please then have a look at "top" and see if anything is consuming a lot of resources and that your memory usage is OK


    Still getting all the notifications after systemd-analyze set-log-level notice. Top shows low CPU and memory usage.
    The reply is currently minimized Show
  • Accepted Answer

    Monday, April 08 2019, 04:34 PM - #Permalink
    Resolved
    0 votes
    @Greg, Is your system still responsive? Log messages can be dropped easily. Also have you raised a ticket at Clearcenter?
    The reply is currently minimized Show
  • Accepted Answer

    Greg
    Greg
    Offline
    Monday, April 08 2019, 05:10 PM - #Permalink
    Resolved
    0 votes
    Nick Howitt wrote:

    @Greg, Is your system still responsive? Log messages can be dropped easily. Also have you raised a ticket at Clearcenter?


    Yes it is responsive but VERY sluggish. I have raised a support ticket.
    The reply is currently minimized Show
  • Accepted Answer

    Tuesday, April 09 2019, 08:28 PM - #Permalink
    Resolved
    0 votes
    @Leon,
    We now have a couple of patches going through and once they sync to the repos, you can update them with a:
    yum update csplugin-events app-base --enablerepo=clearos-updates-testing
    You are looking for csplugin-events-1.2-3.v7 and app-base-2.6.4-1.v7 or app-base-2.6.5-1.v7 (either is OK). We are also looking at patching the audit log but this will be in slow time.

    If you can't wait, as an alternative solution you can add a single line to /etc/pam.d/system-auth-ac:
    session [success=1 default=ignore] pam_succeed_if.so service in sudo quiet uid = 0 ruser = webconfig
    It needs to go above the last two lines so the last section reads:
    session     optional      pam_keyinit.so revoke
    session required pam_limits.so
    -session optional pam_systemd.so
    session optional pam_mkhomedir.so umask=0077
    session [success=1 default=ignore] pam_succeed_if.so service in crond quiet use_uid
    session [success=1 default=ignore] pam_succeed_if.so service in sudo quiet uid = 0 ruser = webconfig
    session required pam_unix.so
    session optional pam_ldap.so

    After making the change there is no need to restart anything. This change should not interfere with the patches.

    You should also be able to clear down the events database with:
    systemctl stop clearsync.service
    rm -f /var/lib/csplugin-events/events.db
    systemctl start clearsync.service
    The reply is currently minimized Show
  • Accepted Answer

    Tuesday, April 09 2019, 10:37 PM - #Permalink
    Resolved
    0 votes
    The reply is currently minimized Show
  • Accepted Answer

    Tuesday, April 09 2019, 11:20 PM - #Permalink
    Resolved
    0 votes
    I logged in to report similar issues on my servers I am getting this:

    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|telldir|ok|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|readdir|ok|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|translate_name|fail (Operation not supported)|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|telldir|ok|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|readdir|ok|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|translate_name|fail (Operation not supported)|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|telldir|ok|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|readdir|ok|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|translate_name|fail (Operation not supported)|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|telldir|ok|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|readdir|ok|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|translate_name|fail (Operation not supported)|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|telldir|ok|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|readdir|ok|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|translate_name|fail (Operation not supported)|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|telldir|ok|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|readdir|ok|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|translate_name|fail (Operation not supported)|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|telldir|ok|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|readdir|ok|
    Apr 9 17:27:15 remote smbd_audit: sharon|192.168.17.188|translate_name|fail (Operation not supported)|

    Literally thousands of logged events. It seems to relate to a Samba update that went in over last weekend early Monday morning and I am getting pages of event notifications.
    Can anyone confirm if this is the same issue as reported by other users here or a different issue?

    Siv
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 07:38 AM - #Permalink
    Resolved
    0 votes
    @Tony,
    It is not an events problem per-se. It is just that is where the problem manifests itself, although if the events system were faster, the problem may have gone away. The real issue seems to be that the webconfig has changed from doing a single "su" to assume root piviliges to doing a "sudo" for every necessary transaction. This is generating a huge amount of transactions - look in the audit log and secure log e.g:
    Apr  9 09:25:53 gateway-h sudo: pam_unix(sudo:session): session opened for user root by (uid=0)
    Apr 9 09:25:53 gateway-h sudo: pam_unix(sudo:session): session closed for user root
    These also go into the events system. It is possible to suppress some of these at source by modding the pam configuration but this is not a program maintained by ClearOS so they are going down the route of blocking the messages from going in the events database. At the same time they are adding an rsyslog filter to drop the messages from the secure log.

    This rsyslog filter is good in that it can easily be added to to get rid of other annoying messages. That is on my to do list.

    There is a residual problem which will be fixed in slow time. There is a huge amount of logging going on in the audit daemon now and the log rotates now every few hours. This will be will be looked at in slow time. It is possible to block the messages by adding a rules file to /etc/audit/rules.d and restarting the audit daemon. To block webconfig messages add:
    -a never,exclude  -F uid=webconfig
    .Root can similarly be blocked but this rather negates the audit daemon so this needs to be stufied. What is really needed to be blocked is where root does a "sudo" and I need to study the manual for that.
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 07:47 AM - #Permalink
    Resolved
    0 votes
    @Graham,
    That is a different issue. Which log file are you seeing that in? Are you using the AD Connector? Or do you have Audit Log enabled in a flexshare?

    Perhaps it would be a good idea to start a new thread and also post the output of "testparm -s"
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 08:04 AM - #Permalink
    Resolved
    0 votes
    Nick Howitt wrote:

    @Graham,
    That is a different issue. Which log file are you seeing that in? Are you using the AD Connector? Or do you have Audit Log enabled in a flexshare?

    Perhaps it would be a good idea to start a new thread and also post the output of "testparm -s"


    Nick,

    This is comes to me each day as a report by email. I haven't looked at any logs as such, I just noticed in the Updates section of the web interface these notifications started being generated after some updates to Samba were applied over the weekend.

    Siv
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 08:10 AM - #Permalink
    Resolved
    0 votes
    Nick Howitt wrote:

    @Graham,
    That is a different issue. Which log file are you seeing that in? Are you using the AD Connector? Or do you have Audit Log enabled in a flexshare?

    Perhaps it would be a good idea to start a new thread and also post the output of "testparm -s"


    [/quote]
    I also am seeing this in the "Events and Notifications" section of the Web GUI:
    User root logged out via sudo 2019-04-10 09:06:16
    User root logged in via sudo 2019-04-10 09:06:15
    User root logged in via sudo 2019-04-10 09:06:13
    User root logged out via sudo 2019-04-10 09:06:13
    User root logged in via sudo 2019-04-10 09:06:00
    User root logged out via sudo 2019-04-10 09:06:00
    User root logged out via sudo 2019-04-10 09:05:57
    User root logged in via sudo 2019-04-10 09:05:56
    User root logged in via sudo 2019-04-10 09:05:01
    User root logged out via sudo 2019-04-10 09:05:01

    There are currenty 10,000 odd entries and it's occurring every second. I can access the server but it's obviously hammering it whatever "it" is.

    Siv
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 08:27 AM - #Permalink
    Resolved
    0 votes
    Nick Howitt wrote:

    @Graham,
    That is a different issue. Which log file are you seeing that in? Are you using the AD Connector? Or do you have Audit Log enabled in a flexshare?

    Perhaps it would be a good idea to start a new thread and also post the output of "testparm -s"


    I can't remote in via SSH as it appears to be not responding I just get "Connection timed out" so can't send you the results of testparm -s

    Though I can get in via the web gui?

    Siv
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 08:48 AM - #Permalink
    Resolved
    0 votes
    @Graham, from earlier in the thread:
    yum update csplugin-events app-base --enablerepo=clearos-updates-testing
    I'll have to get the fixes released tonight. They did not finish compiling before I shut down last night.

    If you do manage to get in by ssh, you can shutdown clearsyncd with a:
    service clearsync stop
    Alternatively, from the webconfig, go Webconfig > System > Settings > Services and shut it down. Once you have done the update, you can start the service again.
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 09:11 AM - #Permalink
    Resolved
    0 votes
    Nick Howitt wrote:

    @Graham, from earlier in the thread:
    yum update csplugin-events app-base --enablerepo=clearos-updates-testing
    I'll have to get the fixes released tonight. They did not finish compiling before I shut down last night.

    If you do manage to get in by ssh, you can shutdown clearsyncd with a:
    service clearsync stop
    Alternatively, from the webconfig, go Webconfig > System > Settings > Services and shut it down. Once you have done the update, you can start the service again.


    In my server I can't see System > Settings > Services. I do get Process Viewer by going to Reports > Performance And Resouces > Process Viewer and in their I get a running process list with a "Kill" button next to each one. I can find clearsyncd in there. When I hit "kill" it says "Confirm delete" and the process ID, will this stop the service or will it actually delete it, I certainly don't want to do that? If it means just stop teh service then I do.

    I won't press "OK" until you get back t me.
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 09:33 AM - #Permalink
    Resolved
    0 votes
    @Graham, I don't think you have a choice. It should just kill the process from running, i.e brutally stop it. It will not delete the service. You should then regain SSH access.

    I've just released the updates, but they won't come through automatically until tonight.
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 09:38 AM - #Permalink
    Resolved
    0 votes
    Nick

    adding a rules file to /etc/audit/rules.d and restarting the audit daemon

    You cannot do this using systemctl on a standard ClearOS system - manual start,stop and restart of auditd.. es ist verboten

    Reboot, or a quick and dirty way...
    1. Edit /etc/systemd/system/multi-user.target.wants/auditd.service
    2. Find the line "RefuseManualStop=yes" and change to "RefuseManualStop=no"
    3. Run "systemctl daemon-reload"
    4. run "systemctl restart auditd"

    auditd is an SSD killer, run most of my ClearOS systems on an SSD - where there is also a standard disk, edit /etc/audit/auditd.conf and log to another directory not on the SSD. For systems with SSD only - crude solution at the moment is to change the line "write_logs = yes" to "write_logs = no" in the conf file..

    Still immune to csplugin-events problems as the /etc/clearsync.d/csplugin-events.conf on all systems is still being deleted by a cron job. A CPU being driven to nearly 100% by this event file is not a trivial problem...
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 10:08 AM - #Permalink
    Resolved
    0 votes
    Tony Ellis wrote:

    Nick

    adding a rules file to /etc/audit/rules.d and restarting the audit daemon

    You cannot do this using systemctl on a standard ClearOS system - manual start,stop and restart of auditd.. es ist verboten
    Are you sure? I've been doing it yesterday and today to troubleshoot adding rules to /etc/audit/rules.d.

    There may be a particular auditd rule you can enable which may enforce the "RefuseManualStop" (/usr/share/doc/audit-2.8.4/rules), but with a system I installed a couple of days ago, I can stop/start/restart.
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 10:35 AM - #Permalink
    Resolved
    0 votes

    [root@danda rules.d]# systemctl stop auditd
    Failed to stop auditd.service: Operation refused, unit auditd.service may be requested by dependency only (it is configured to refuse manual start/stop).
    See system logs and 'systemctl status auditd.service' for details.

    Same on every system - all 7.x community...
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 10:51 AM - #Permalink
    Resolved
    0 votes
    Tony Ellis wrote:


    [root@danda rules.d]# systemctl stop auditd
    Failed to stop auditd.service: Operation refused, unit auditd.service may be requested by dependency only (it is configured to refuse manual start/stop).
    See system logs and 'systemctl status auditd.service' for details.

    Same on every system - all 7.x community...
    Here is a nice bug for you. Instead try:
    service auditd stop


    [root@gateway-h ~]# service auditd start
    Redirecting to /bin/systemctl start auditd.service
    [root@gateway-h ~]# systemctl stop auditd
    Failed to stop auditd.service: Operation refused, unit auditd.service may be requested by dependency only (it is configured to refuse manual start/stop).
    See system logs and 'systemctl status auditd.service' for details.

    [root@gateway-h ~]# service auditd stop
    Stopping logging: [ OK ]
    I was being lazy and using the service command!
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 10:53 AM - #Permalink
    Resolved
    0 votes
    Nick Howitt wrote:

    @Graham, I don't think you have a choice. It should just kill the process from running, i.e brutally stop it. It will not delete the service. You should then regain SSH access.

    I've just released the updates, but they won't come through automatically until tonight.


    Nick I managed to kill clearsyncd but still can't get in via SSH it just keeps timing out? The users don't appear to be having any issues so I think I will wait and see if the update gets through tonight, or do I have to run the command:

    yum update csplugin-events app-base --enablerepo=clearos-updates-testing


    Before I can get your fixes?

    I looked again at the processes and clearsyncd has turned back on again. It did get killed through the web GUI but has started back up again.

    Siv
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 11:03 AM - #Permalink
    Resolved
    0 votes
    @Siv,
    I've released the programs into clearos-updates so they will update tonight. You could try Webconfig > Cloud > Updates > Software Updates and see if that pull them down. I can't test as I've updated my systems. If it shows them but does not give you the chance to update, try switching to a manual update (disable "Automatic Updates").

    If you can't update, I suggest you shut down the webconfig on your PC or avoid the screens with a lot of ajax. I don't know which they are, but any with fields that update as you watch the screen, so certainly avoid the IP Settings screen.
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 11:23 AM - #Permalink
    Resolved
    0 votes
    Nick - still the same on the latest Fedora, just tried it... thanks for the workaround...

    Maybe "service" is classified under "may be requested by dependency only"
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 11:43 AM - #Permalink
    Resolved
    0 votes
    Tony Ellis wrote:

    Nick - still the same on the latest Fedora, just tried it... thanks for the workaround...

    Maybe "service" is classified under "may be requested by dependency only"
    I suspect it is just an upstream bug which someone noticed and patched and FC is way ahead of ClearOS/Centos/RHEL. We are just using the el7 version of systemd unmodified.

    [edit]
    auditd changed to systemd
    [/edit]
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 11:51 AM - #Permalink
    Resolved
    0 votes
    Sorry - wasn't clear enough. Fedora 29 still the same result as with ClearOS/CentOS 7.x - systemctl doesn't work without the edit to the conf file, service does work regardless...
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 01:49 PM - #Permalink
    Resolved
    0 votes
    Nick Howitt wrote:

    @Siv,
    I've released the programs into clearos-updates so they will update tonight. You could try Webconfig > Cloud > Updates > Software Updates and see if that pull them down. I can't test as I've updated my systems. If it shows them but does not give you the chance to update, try switching to a manual update (disable "Automatic Updates").

    If you can't update, I suggest you shut down the webconfig on your PC or avoid the screens with a lot of ajax. I don't know which they are, but any with fields that update as you watch the screen, so certainly avoid the IP Settings screen.


    Nick,
    I finally did manage to get in via SSH and was able to run the command:

    yum update csplugin-events app-base --enablerepo=clearos-updates-testing


    So I am hopeful that your updates will get through and quieten things down again.

    I will watch to see if the other messages come up in the emailed report I get sent at the end of the day that I initially reported as I thought they were related to this. If they do I will raise them as a new thread in these forums.

    As always thanks for all you do Nick.
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 02:01 PM - #Permalink
    Resolved
    0 votes
    @Nick,
    I just checked my system and no outstanding updates, the following were the updates that were applied most recently and seemed to trigger all sorts of log activity:
     Package	Action	Date/Time
    elfutils-default-yama-scope-0.172-2.el7 Updated Apr 08, 01:02:54
    linux-firmware-20180911-69.git85c5d90.el ... Updated Apr 07, 05:48:51
    lm_sensors-libs-3.4.0-6.20160601gitf9185 ... Updated Apr 07, 05:48:41
    samba-python-4.8.3-4.3.v7 Updated Apr 07, 05:48:41
    grub2-2.02.1-0.76.v7.1 Updated Apr 07, 05:48:40
    samba-winbind-clients-4.8.3-4.3.v7 Updated Apr 07, 05:48:40
    samba-client-4.8.3-4.3.v7 Updated Apr 07, 05:48:39
    samba-dc-4.8.3-4.3.v7 Updated Apr 07, 05:48:39
    libsmbclient-4.8.3-4.3.v7 Updated Apr 07, 05:48:38
    samba-dc-libs-4.8.3-4.3.v7 Updated Apr 07, 05:48:38
    samba-winbind-4.8.3-4.3.v7 Updated Apr 07, 05:48:38
    samba-winbind-modules-4.8.3-4.3.v7 Updated Apr 07, 05:48:35
    grub2-pc-2.02.1-0.76.v7.1 Updated Apr 07, 05:48:34
    grub2-pc-modules-2.02.1-0.76.v7.1 Updated Apr 07, 05:48:34
    grub2-tools-extra-2.02.1-0.76.v7.1 Updated Apr 07, 05:48:33
    samba-4.8.3-4.3.v7 Updated Apr 07, 05:48:33
    samba-common-tools-4.8.3-4.3.v7 Updated Apr 07, 05:48:27
    grub2-tools-2.02.1-0.76.v7.1 Updated Apr 07, 05:48:26
    grub2-tools-minimal-2.02.1-0.76.v7.1 Updated Apr 07, 05:48:26
    grub2-common-2.02.1-0.76.v7.1 Updated Apr 07, 05:48:25
    samba-libs-4.8.3-4.3.v7 Updated Apr 07, 05:48:25
    samba-client-libs-4.8.3-4.3.v7 Updated Apr 07, 05:48:24
    samba-common-libs-4.8.3-4.3.v7 Updated Apr 07, 05:48:23
    libwbclient-4.8.3-4.3.v7 Updated Apr 07, 05:48:22
    samba-common-4.8.3-4.3.v7 Updated Apr 07, 05:48:21
    mesa-libwayland-egl-17.2.3-8.20171019.el ... Erased Apr 06, 02:07:11
    iwl3160-firmware-22.0.7.0-69.el7 Updated Apr 06, 02:02:59
    iwl3945-firmware-15.32.2.9-69.el7 Updated Apr 06, 02:02:59
    iwl6050-firmware-41.28.5.1-69.el7 Updated Apr 06, 02:02:59
    iwl100-firmware-39.31.5.1-69.el7 Updated Apr 06, 02:02:58
    iwl2000-firmware-18.168.6.1-69.el7 Updated Apr 06, 02:02:58
    iwl6000-firmware-9.221.4.1-69.el7 Updated Apr 06, 02:02:58
    iwl7260-firmware-22.0.7.0-69.el7 Updated Apr 06, 02:02:58
    iwl1000-firmware-39.31.5.1-69.el7 Updated Apr 06, 02:02:57
    iwl135-firmware-18.168.6.1-69.el7 Updated Apr 06, 02:02:57
    iwl5150-firmware-8.24.2.2-69.el7 Updated Apr 06, 02:02:57
    iwl6000g2b-firmware-17.168.5.2-69.el7 Updated Apr 06, 02:02:57
    emacs-filesystem-24.3-22.el7 Updated Apr 06, 02:02:56
    iwl105-firmware-18.168.6.1-69.el7 Updated Apr 06, 02:02:56
    iwl2030-firmware-18.168.6.1-69.el7 Updated Apr 06, 02:02:56
    iwl4965-firmware-228.61.2.24-69.el7 Updated Apr 06, 02:02:56
    iwl5000-firmware-8.83.5.1_1-69.el7 Updated Apr 06, 02:02:56
    iwl6000g2a-firmware-17.168.5.3-69.el7 Updated Apr 06, 02:02:56
    dialog-1.2-5.20130523.el7 Updated Apr 06, 02:02:55
    dosfstools-3.0.20-10.el7 Updated Apr 06, 02:02:55
    dmidecode-3.1-2.el7 Updated Apr 06, 02:02:53
    iw-4.3-2.el7 Updated Apr 06, 02:02:53
    iptraf-ng-1.1.4-7.el7 Updated Apr 06, 02:02:52
    libgomp-4.8.5-36.el7 Updated Apr 06, 02:02:52
    augeas-libs-1.4.0-6.el7_6.1 Updated Apr 06, 02:02:51


    Because the initial errors I posted seemed to relate to Samba and seemed to start around the time the 7th April updates were being applied i have assumed the two things are related.

    One other thing that occurred was the Windows machines that had mapped links to Flexshares on the server suddenly would not connect and I had to restart the server to get them to connect up again.

    Siv
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 02:07 PM - #Permalink
    Resolved
    0 votes
    @Nick,
    When I ran the command it came up with "No packages marked for update":

    yum update csplugin-events app-base --enablerepo=clearos-updates-testing
    Loaded plugins: clearcenter-marketplace, fastestmirror
    ClearCenter Marketplace: fetching repositories...
    Loading mirror speeds from cached hostfile
    * clearos: www.mirrorservice.org
    * clearos-centos: download3.clearsdn.com
    * clearos-centos-sclo-rh: download3.clearsdn.com
    * clearos-centos-updates: download3.clearsdn.com
    * clearos-contribs: www.mirrorservice.org
    * clearos-epel: download3.clearsdn.com
    * clearos-fast-updates: download3.clearsdn.com
    * clearos-infra: www.mirrorservice.org
    * clearos-updates: www.mirrorservice.org
    * clearos-updates-testing: www.mirrorservice.org
    * private-clearcenter-dyndns: download1.clearsdn.com:80
    * private-clearcenter-roundcubemail: download1.clearsdn.com:80
    clearos-updates-testing | 3.5 kB 00:00
    (1/2): clearos-updates-testing/7/updateinfo | 96 B 00:00
    (2/2): clearos-updates-testing/7/primary_db | 736 kB 00:00

    No packages marked for update

    [root@remote ~]# service clearsync start
    Redirecting to /bin/systemctl start clearsync.service


    Siv
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 02:23 PM - #Permalink
    Resolved
    0 votes
    The update that you should be running is csplugin-events-1.2-3.v7

    http://koji.clearos.com/koji/buildinfo?buildID=2618

    You can look at what version you are running by using:

    rpm -qi csplugin-events

    If you are not running this version, run the following to clean up your yum cache:

    yum --enablerepo=* clean all

    As you can see from koji, this package is in clearos-updates. If you are running ClearOS Community you will see this by just doing a 'yum update' after doing the 'yum --enablerepo=* clean all'

    If you are running ClearOS Business and have installed these updates already, then you will need to enable the clearos-udpates repo in order to grab this update:

    yum --enablerepo=clearos-updates update csplugin-events
    The reply is currently minimized Show
  • Accepted Answer

    Wednesday, April 10 2019, 02:26 PM - #Permalink
    Resolved
    0 votes
    That is weird. Just a plain old "yum update" should work as I moved them into the clearos-update this morning. The --enablerepo bit is irrelevant now and the yum command should work with or without it. Can I suggest you leave the clearsync service disabled for the moment (or until you get the updates) and then do a:
    yum clean all && yum update
    The reply is currently minimized Show
Your Reply