Does anyone know how to reset the ldap database, since the last update all I get is accounting offline errors, tried the fixes thats been posted in the past to no avail.
I've even reinstalled clear 7 for it to fail again after the updates and a reboot.
Is there a way to reinitialise like from when you first install? I only have 4 users so redoing them is not a problem .
I've even reinstalled clear 7 for it to fail again after the updates and a reboot.
Is there a way to reinitialise like from when you first install? I only have 4 users so redoing them is not a problem .
Share this post:
Responses (23)
-
Accepted Answer
Just a quick follow up to the 'Account is offline' issue...my hope is that this issue is now permanently resolved with an update to app-ldap (version 2.1.7-1) with the fix suggested in the bug tracker here.
I say 'hope' because it was an intermittent bug..however, the customer I've been working with used to see it pop up 3-4 times a week, and has now had over three weeks without seeing it once.
B. -
Accepted Answer
We made some huge progress today in that we were able to find a way to duplicate it.
It all depends on the order of the systemd startup services.
If webconfig comes up too soon before network.service (note, the systemd configlet for webconfig already has an 'after=network.target'), we can make it fail in the same way you guys have reported here (the post boot group).
However, the only way we've been able to make it fail is to hack the systemd configlet for the webconfig service to purposefully start up before. We cannot make it fail with the default systemd files.
If you want to prove you have an issue, run:
systemd-analyze plot > /tmp/startup.svg
Paste the SVG output here or send to [email protected].
We're interested to know how close together the webconfig.service and network.service services are.
As for a fix....a bit of a hack, but we believe editing:
/usr/libexec/webconfig/prestart.sh
Add close to the top of that file, add a "sleep 5":
#!/bin/sh
sleep 5
# Environment
KEY="/usr/clearos/sandbox/etc/httpd/conf/server.key"
...
..
.
This should delay Webconfig service from starting from after network.service has done all its 'bits' and be OK.
The true fix is to see what's really going on the the PHP ldap library...now that we can reproduce, we'll try and investigate and see if we can suggest an upstream patch.
Cheers,
Ben -
Accepted Answer
-
Accepted Answer
-
Accepted Answer
-
Accepted Answer
systemctl smb stop
systemctl nmb stop
systemctl winbind stop
systemctl nscd stop
systemctl stop smb
systemctl stop nmb
systemctl stop winbind
systemctl stop nscd
systemctl stop nslcd
systemctl stop slapd
Then do
rm /var/clearos/accounts/initialized /var/clearos/accounts/config /var/clearos/ldap/initialized /var/clearos/openldap/config.php /var/clearos/mode/mode.conf /var/clearos/samba/init* -fv
cd /var/lib/ldap
rm -df *
then I did
yum reinstall webconfig-php-ldap-5.4.16-36.v7.1.x86_64 app-ldap-core-2.1.6-1.v7.noarch app-openldap-core-2.1.7-1.v7.noarch app-openldap-directory-core-2.1.6-1.v7.noarch openldap-clients-2.4.40-8.v7.x86_64 openldap-2.4.40-8.v7.x86_64 php-ldap-5.4.16-36.el7_1.x86_64 nss-pam-ldapd-0.8.13-8.el7.x86_64 openldap-servers-2.4.40-8.v7.x86_64 freeradius-ldap-3.0.4-6.el7.x86_64
Then reboot,
DO NOT install/initialise the Directory server it will fail again after you've initialised. That's what was causing it on my end, There has been updates since so dont know if its a unknown/weird issues that's now fixed. -
Accepted Answer
-
Accepted Answer
Rodrigo Lozada wrote:
Thanks for the info.
One question, what do you mean by "Dot not install the directory server" ?
That is precisely what I'm trying to do. I want to manage the users on the system but it fails.
Sorry if I misunderstood you.
I should have been more clear, sorry.
I mean the "Directory server app" via the market place/store.
The accounting system uses ldap fine, as soon as I installed the "Directory server app" from the market in caused problems for me and ended up with "accounting system offline"
If you don't need remote access to the directory it's not needed
HTH -
Accepted Answer
pete910 wrote:
systemctl smb stop
systemctl nmb stop
systemctl winbind stop
systemctl nscd stop
systemctl stop smb
systemctl stop nmb
systemctl stop winbind
systemctl stop nscd
systemctl stop nslcd
systemctl stop slapd
Then do
rm /var/clearos/accounts/initialized /var/clearos/accounts/config /var/clearos/ldap/initialized /var/clearos/openldap/config.php /var/clearos/mode/mode.conf /var/clearos/samba/init* -fv
cd /var/lib/ldap
rm -df *
then I did
yum reinstall webconfig-php-ldap-5.4.16-36.v7.1.x86_64 app-ldap-core-2.1.6-1.v7.noarch app-openldap-core-2.1.7-1.v7.noarch app-openldap-directory-core-2.1.6-1.v7.noarch openldap-clients-2.4.40-8.v7.x86_64 openldap-2.4.40-8.v7.x86_64 php-ldap-5.4.16-36.el7_1.x86_64 nss-pam-ldapd-0.8.13-8.el7.x86_64 openldap-servers-2.4.40-8.v7.x86_64 freeradius-ldap-3.0.4-6.el7.x86_64
Then reboot,
DO NOT install/initialise the Directory server it will fail again after you've initialised. That's what was causing it on my end, There has been updates since so dont know if its a unknown/weird issues that's now fixed.
This question takes me two days. Then I found if you modify the .ini file in /usr/clearos/sandbox/etc/php.d let php service reload, everything will be fine untill next reboot.
So ,I think the reason is the service startup order. -
Accepted Answer
-
Accepted Answer
I'm still not sure if this issue is behind us or not...one one hand, looks like the update (which closed open LDAP connections when a PHP class is done with them) looks to have permanently solved one customer I worked with.
However, yesterday we got a report of "Account is offline" issue whenever the server was rebooted...I can't think that open connections could be an issue when a server is rebooted, so I am at a loss to explain yesterday's report...though I could not and did not confirm it.
If anyone coming across this thread has this issue and can reproduce it repeatedly, can you I ask you to do one of two things (or both);
1. In /usr/clearos/apps/ldap/libraries/LDAP_Client.php
Look for the function/line:
protected function _bind($mode)
{
And add to the top of that function so it looks like this:
protected function _bind($mode)
{
clearos_profile(__METHOD__, __LINE__);
ldap_set_option(NULL, LDAP_OPT_DEBUG_LEVEL, 7);
Don't restart webconfig...that will temporarily fix your issue...just reload the page that uses the LDAP PHP library (eg. the users page @ /app/users).
Once done, grab your /var/log/webconfig/error_log file and post that last 20+ lines of debug output while you're seeing the "Account is offline" message.
2. Create a support ticket here and I will work with you free of charge, regardless of what edition you're running if you are OK with spending some cycles with me to reproduce the issue and get this debug off the system in hopes of moving this bug off permanently.
Thanks,
Ben -
Accepted Answer
Hi Ben
I don't know if this helps or not, but I now have 2,000 lines in the error log repeating this:-
ldap_err2string
ldap_bind_s
ldap_simple_bind_s
ldap_sasl_bind_s
ldap_sasl_bind
ldap_send_initial_request
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP 127.0.0.1:389
ldap_connect_to_host: getaddrinfo failed: Address family for hostname not supported
ldap_err2string
ldap_bind_s
ldap_simple_bind_s
ldap_sasl_bind_s
ldap_sasl_bind
ldap_send_initial_request
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP 127.0.0.1:389
ldap_connect_to_host: getaddrinfo failed: Address family for hostname not supported
ldap_err2string
This is a brand new COS 7.2 (business - evaluation) install. Installed three days ago behind a COS 6.5 box (which provides the 7.2 setup with its IP address) the only changes I have made in the 7.2 setup are:-
Installed Webmin and applied bug file 9541 workaround (reload fail2ban).
Steve -
Accepted Answer
Well I removed the fail2ban reload just in case but no joy. I now have an error log with over 8,000 lines of the same as above. Just before the repeating lines the log shows:
[Tue Aug 02 14:34:37.831914 2016] [suexec:notice] [pid 808] AH01232: suEXEC mechanism enabled (wrapper: /usr/clearos/sandbox/usr/sbin/suexec)
[Tue Aug 02 14:34:37.849228 2016] [ssl:warn] [pid 808] AH02292: Init: Name-based SSL virtual hosts only work for clients with TLS server name indication support (RFC 4366)
[Tue Aug 02 14:34:37.883966 2016] [auth_digest:notice] [pid 808] AH01757: generating secret for digest authentication ...
[Tue Aug 02 14:34:37.884785 2016] [lbmethod_heartbeat:notice] [pid 808] AH02282: No slotmem from mod_heartmonitor
[Tue Aug 02 14:34:37.887015 2016] [ssl:warn] [pid 808] AH02292: Init: Name-based SSL virtual hosts only work for clients with TLS server name indication support (RFC 4366)
[Tue Aug 02 14:34:39.600969 2016] [mpm_prefork:notice] [pid 808] AH00163: Apache/2.4.6 (ClearOS) OpenSSL/1.0.1e-fips configured -- resuming normal operations
[Tue Aug 02 14:34:39.601009 2016] [core:notice] [pid 808] AH00094: Command line: '/usr/sbin/webconfig -D FOREGROUND'
ldap_url_parse_ext(ldap://localhost/)
ldap_init: trying /etc/openldap/ldap.conf
ldap_init: using /etc/openldap/ldap.conf
ldap_init: HOME env is NULL
ldap_init: LDAPCONF env is NULL
ldap_init: LDAPRC env is NULL
ldap_create
ldap_url_parse_ext(ldap://127.0.0.1)
ldap_bind_s
ldap_simple_bind_s
ldap_sasl_bind_s
ldap_sasl_bind
ldap_send_initial_request
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP 127.0.0.1:389
ldap_connect_to_host: getaddrinfo failed: Address family for hostname not supported
ldap_err2string
Hopefully this might help track down the issue. Anyway I am going to reinstall and see what happens, I'll let you know if the problem reoccurs.
Steve -
Accepted Answer
Thanks Stephen,
If you edit /usr/lib/systemd/system/webconfig.service
and change this line:
After=network.target remote-fs.target nss-lookup.target
To:
After=network.target remote-fs.target nss-lookup.target dnsmasq.service
And you reboot, does it resolve the 'Offline' issue on reboot? Not saying this is an eloquent or 'goto' solution...just trying to understand which race condition we're up against in the boot up sequence to get around this.
B. -
Accepted Answer
Made the change and the problem is still there, this was the latest error log output:
[Tue Aug 02 15:11:18.762603 2016] [mpm_prefork:notice] [pid 808] AH00170: caught SIGWINCH, shutting down gracefully
[Tue Aug 02 15:11:40.616333 2016] [suexec:notice] [pid 729] AH01232: suEXEC mechanism enabled (wrapper: /usr/clearos/sandbox/usr/sbin/suexec)
[Tue Aug 02 15:11:40.667662 2016] [ssl:warn] [pid 729] AH02292: Init: Name-based SSL virtual hosts only work for clients with TLS server name indication support (RFC 4366)
[Tue Aug 02 15:11:40.726412 2016] [auth_digest:notice] [pid 729] AH01757: generating secret for digest authentication ...
[Tue Aug 02 15:11:40.727738 2016] [lbmethod_heartbeat:notice] [pid 729] AH02282: No slotmem from mod_heartmonitor
[Tue Aug 02 15:11:40.731520 2016] [ssl:warn] [pid 729] AH02292: Init: Name-based SSL virtual hosts only work for clients with TLS server name indication support (RFC 4366)
[Tue Aug 02 15:11:42.066963 2016] [mpm_prefork:notice] [pid 729] AH00163: Apache/2.4.6 (ClearOS) OpenSSL/1.0.1e-fips configured -- resuming normal operations
[Tue Aug 02 15:11:42.067006 2016] [core:notice] [pid 729] AH00094: Command line: '/usr/sbin/webconfig -D FOREGROUND'
ldap_url_parse_ext(ldap://localhost/)
ldap_init: trying /etc/openldap/ldap.conf
ldap_init: using /etc/openldap/ldap.conf
ldap_init: HOME env is NULL
ldap_init: LDAPCONF env is NULL
ldap_init: LDAPRC env is NULL
ldap_create
ldap_url_parse_ext(ldap://127.0.0.1)
ldap_bind_s
ldap_simple_bind_s
ldap_sasl_bind_s
ldap_sasl_bind
ldap_send_initial_request
ldap_new_connection 1 1 0
ldap_int_open_connection
ldap_connect_to_host: TCP 127.0.0.1:389
ldap_connect_to_host: getaddrinfo failed: Address family for hostname not supported
ldap_err2string
Then repeating the same as before
Steve -
Accepted Answer
-
Accepted Answer
-
Accepted Answer
Thanks Stephen,
See how your nework.service takes a while to start (5.119s) and the completion ends past the end time of the webconfig.service. You are the second instance where we have confirmed this and in both cases, you guys are the ones seeing the 'Account is Offline' message on boot.
I resolved this earlier today with another customer by editing:
/usr/lib/systemd/system/webconfig.service
And modifying the 'After' line so it waits until after network.service instead of network.target and rebooting.
Your confirmation would be another step towards a permanent fix, however, I'd like to better understand the repercussions and need to consult with Peter B (who's currently away until next week). Darryl's also going to spend some cycles to see why this PHP library behaives like it does, because ultimately, anything we do in systemd is a work-around...a solution is trickier and in likely to be as a PHP patch upstream.
B. -
Accepted Answer
Cheers Ben
As it may be something connected with network.service, I just thought I would mention that my install sits on virtual machine and gets its IP via dhcp from a COS6.5 virtual machine on the same server. The two setups are using similar hostnames e.g. COS6.5 = server.lan and COS7.2 = second.server.lan.
The only other thing is that I have just had a look at the boot.log and found this in it:
[[32m OK [0m] Started ACPI Event Daemon.
[[1;31mFAILED[0m] Failed to start Arpwatch ethernet/IP address tracking on ens160.
See 'systemctl status [email protected]' for details.
I'm not sure if this helps or hinders or is nothing at all.
Steve. -
Accepted Answer
-
Accepted Answer
There is some interesting stuff on network.target on the web, including this link where they say it should only really be used for shutdown. Waiting for it won't guarantee slapd an IP address if it is trying to publish its services. I don't think network-online.target looks like much help either as what happens if no interfaces come up e.g. during an offline installation. Would that then hang the start up?
Does the network service rely on external factors (such as obtaining DHCP leases)? In that case its start up time could be unpredictable.
I'm just throwing some thoughts into the pot and could be well wide of the mark. -
Accepted Answer
By way of follow up in case it helps someone else
Based on Ben's suggestions
In
/usr/lib/systemd/system/webconfig.service
I changed the line starting with
After=network.target
to
After=network.service remote-fs.target nss-lookup.target
And the 'Account System is offline' issue went away after a reboot.
Steve. -
Accepted Answer
I had this issue today after adding another external NIC to the server. Turns out it was caused by me forgetting to untick the use automatic DNS and Clearos defaulting to the DNS from the new ADSL router and note my internal DNS of AD. Simple little thing, much frustration.
Hopefully that may save someone else some time.
Please login to post a reply
You will need to be logged in to be able to post a reply. Login using the form on the right or register an account if you are new here.
Register Here »