All in one night I took my Nagios server from running Fedora 16 to CENTOS 6.5, Updated XenCenter to the latest version 6.2.2 and updated XenServer to 6.2.0 SP1 and solved and issue where my iSCSI target was failing in FreeNAS. I had only planned to get Nagios up and going on the CENTOS server but upon completion of getting the Nagios server configured with the basic systems and monitoring I noticed this nice red X on one of my iSCSI targets in XenServer.

                First the Nagios server I had copied my configuration files to my NFS drives prior to blowing away the Fedora 16 OS, so once I had CENTOS 6.5 installed I connected back to the NFS share and made the appropriate changes to my configuration files to point to my new servers. Now that my web server and gaming server plus a few other systems are now in my XenServer virtual environment with different names I just had to go and make the changes in the Nagios configuration files.

                I was about to happily grab a book and retire to my bedroom when the iSCSI issue on my XenServer came to my attention. I looked and it said it was missing 2 of my multipathing connections. I checked the connectivity and that was fine as it should have been otherwise several other storage volumes should have shown the same issue.

                Well logging into my second FreeNAS node I noticed that it was throwing out some errors and I could not login to do anything. I captured some of the errors with this really catching my eye: “istgt_iscsi.c:5551:worker:***ERROR***pthread_create() failed”. I found a number of people having this issue in the FreeNAS forums  with memory issues caused by ZFS volumes. Last time I checked this specific node having the issues was carrying less of a load and not using near as much memory as the other node not giving errors.

                I finally decided to shutdown my environment and troubleshoot that FreeNAS node. What I found was that there were some CPU errors with the codes of “8121” (temp trip, a minor error) & “84ff (CPU system log full a major error)”. I went into the bios of the system and cleared the error logs and the system then that FreeNAS node came up just fine. When I placed the loads back on both FreeNAS systems they both are once again working like troopers. I do need to tackle temperature issues in my computer room though.

      While I was going through the logs of  XenServer I had noticed the XenCenter update and the SP1 update and figured since I had created my own little maintenance windows I may as well do the updates now! Updating the XenCenter of course was no issues just run the “.msi” file and it took about 2 minutes. Installing XenServer XS62ESP1took a bit more time but went smoothly as well. The SP seemed to have some support for Windows 8.1 desktop and W2012 Server version plus some drivers so it should be a big plus and I will upgrade the VM XenTools later especially on my W2012 server evaluation server.

                I also took the time to switch back to my original master for my XenServer pool cluster.

                I also reconfigured one of my router’s that I had already installed DD-WRT on for one of my static Ip’s, what a night!

So tonight rebuilt Nagios server with new Linux version of Centos 6.5, updated my Virtual environment and corrected some issues on my FreeNAS. All I wanted to do was my Nagios server but a productive night with no sleep. I’m going to bed and will unplug the land line to the bedroom!