March 24, 2016
by dhoytt
0 comments
My UPS’s I updated and put into working order a couple of weekends ago are operating just fine though I may get one more just to spread the load out a bit better.
I have 2 Liebert GXT2-1500rt120 and 1 GXT2-1000 each with and expansion battery pack
.
January 16, 2016
by dhoytt
0 comments
I have no reason why this happened but after installing a slew of updates on my Centos 7 Web Server some of the firewall settings I had allowing forwarding of ports for my web server were closed. For some reason my default routes were pointing to the wrong network interfaces after the reboot as well. I didn’t notice sine I had turned off my audible notifications plus my workstations in my office displayed my sites just fine.
When I went for my workout at the gym today I noticed my stream was working from my music server but I couldn’t get my site up though to see what was coming up next on my station. Then O looked at my text messages and saw the notice that http was down.
When home from the gym checked my httpd process that was up then made sure my router was forwarding and then checked my firewall and http/https were not open! Fixed that restarted httpd and still no love for my web site. rechecked everything and then peeked at the routing table and the deafult route was pointing at the wrong interface so I fixed that in the script files taking default route definition out of the secondary interface and restarted network then httpd and presto everything works!
Never had a patch update no matter how big do that but stuff happens, plus I did see a few inconsistencies in hosts file. All working now and back to the rest of my Saturday!
August 23, 2015
by dhoytt
0 comments
I cleaned up the front yard and did some work in the backyard and the vegetable gardens in the backyard as well. I’m getting ready for the late season for when it starts to cool a bit and some vegetables grow better in the Sacramento/San Joaquin valley Here is some of what is going on in the vegetable garden at this time
Here is some purple okra, Collard greens, purple basil, with sweet basil in view as well:
Here are some peppers with stevia in the foreground:
Here’s some jalepaeno peppers going strong:
In the background is a tomato plant I had to rescue and stake up today plus some more stevia and mint in the foreground:
Here’s a top and bottom shot of my eggplants:
Some chives:
Some new straight neck squash after picking quite a few this week:
Overall of the west side of the vegetable garden with mint plants providing protection in the front concealing some of the other vegetables:
Just a clump of beef eater tomatoes, chives, mint, basil and yellow pear tomatoes:
More stevia:
East side of the garden with some new seeds I planted to get a start in planted pots first and then I will transplant them into the garden. Note the swiss chard standing strong that I replanted from last year.
August 23, 2015
by dhoytt
0 comments
I updated the WordPress blog software after I took some manual backups of the database. Everything went fine but of course I had to go in and manually put back in some customization’s I have made for the sites that the theme updates erased but that went smoothly.
I spread out my virtual machines more evenly in my XenServer environment and reset the HA settings on my XenServer pool. I had a couple of months ago replaced a motherboard that had failed on one of my XenServers and decided to double the RAM from a system I wasn’t using. That system kept randomly restarting as I increased the load. Finally earlier this week I ran memory check from memtest.org and received a flood of errors deep into the test (about 60% in).
I took the memory of the same speed but a different manufacture out of my XenServer system I had repaired then reran the memory test and everything was fine. This memory I took out ran the memory test fine in the other system as well but together the two memory modules from different manufactures didn’t perform together well. Had this went well I would have updated the other system to this memory level as well.
Now my environment is purring along and seeming very stable.
July 24, 2015
by dhoytt
1 Comment
I have been having sporadic Internet connectivity issues lately and today noticed this happened several times Finally about 30 minutes ago I get texted from monitoring I have on my site notifying me my site is down, I discover I cannot contact my router and it’s literally powered off. I look over the router and the power switch is no longer depressed and will not stay depressed to keep the power on!
I had to use duct tape since electrical tape wasn’t quite adhesive enough and some folded paper to keep the power switch depressed with electrical tape keeping the edges down and smooth.
Then went on Internet and found this is a consistent issue with all ASUS routers. I have already had my share of issues from the “kernel: VFS: file-max Limit 22684 reached” error which causes me to have to occasionally reboot since that simple kernel parameter cannot be changed in ASUS’ firmware. Yes I know I can use “dd-wrt” or “tomatoe” open source router firmware but I have the VPN working nicely with this router.
Besides this recent power button issue and the kernel parameter issue I’m also not a huge fan of their interface, specifically the “Virtual Server/Port Forwarding” interface is clunky and a time sync! When I clone or switch web servers I like to just point to the new server in my environment. With ASUS I have to totally delete each prior entry and add a totally new entry for that port forwarding to the new system! Every other router interface you just change a few alphanumeric characters and apply and your done.
Time to get a new router GRRR!!!!
July 11, 2015
by dhoytt
0 comments
Well I moved my website over to a new OS Centos 7 from Fedora 21. Centos seems to be handing my enviornment better with the XFS filesystem than Fedora has been with the ext4 filesystem. I am also still making changes to the infrastructure and updated some features on the “House of Beats” (http://dhoytt.com/snake-ice-radio-blog/now-playing/) blog on this site.
I even gave the system a bit more RAM.
June 13, 2015
by dhoytt
0 comments
Can’t wait to taste this home made concoction made from some mint plants running amok in the garden and some stevia plants I added for a bit of sweet. I didn’t place a lot of stevia in the mix but will add more later to individual cup of tea to sweeten it up a bit.
This is the tea brewing in the sun with some mint in the background I let grow a bit unchecked:
This is the tea done steeping in the sun all day (note tea basket with mint & stevia on bottom of container):
This shows a couple of patches of mint in my garden that is full of mint:
This shows a couple of patches of stevia in my garden:
May 30, 2015
by dhoytt
0 comments
As I was migrating data from one NAS to another this week my XenServer pool master had a memory failure while I was away at work. Well after isolating the DIMM with the issue I also had to go in and change the bios settings for the fans. I noticed that the IPMI log files had numerous references to the fans and noted not all internal fans were set at optimal server fan settings.
Once I had the XenServer back in the rack cabled up it I could not see it boot up and had to switch it to another port on the KVM to see it boot and make subsequent updates. I’ll have to test the KVM port later but I suspect all is okay since other ports on that KVM are fine and it’s connected with my other KVM that also works fine. The other issue that manifested itself was that the XenServer that had the memory issue could no longer see storage from the NAS system I was moving the data off of.
I ended up doing the following to get my storage seen on my XenServer host that had the memory error:
I still want to reconfigure my NAS storage to make it more efficient and still have a lot of work to be done still configuring my applications that serve up my website. I just performed OS updates of the VM I have the site on and noticed Xen library updates which I hope helps in Fedora. I notice Centos behaves a little better in XenServer environment so I may switch this site to a Centos VM.
Ahh well time for a late breakfast!
May 7, 2015
by dhoytt
0 comments
This week since Sunday evening I have been recovering my web site step by step and one of my NAS systems as well. This is why I have redundancy of systems and cloned systems. Still I really didn’t have my backups running the way I wanted or I would have been up right away.
I moved a few systems over each evening to my other NAS and made backups of databases and restored and copied files and made snapshots at every step so I would not have to start over. I was actually running my website on the cloned system so I revived the original system restored the MySQL backups from about a week ago and most everything was back. I still have a few plugins to install for the website and make a few tweaks. The music server being separate was up the whole time so music was still streaming.
As far as repairing the NAS the boot flash drive became corrupted. As long as I didn’t reboot it should have been fine and was since most of the executables were in memory at that point. I saved the configuration to another system, moved the virtual systems over to storage on my other NAS and reinstalled to another flash drive. I uploaded the saved configuration and everything looks great!
April 1, 2015
by dhoytt
0 comments
Well I finally updated my second NAS to FreeNAS 9.3 and it was not straightforward even though it has the exact same hardware build as the first NAS it crashed hard when I attempted the update and totally corrupted my FreeNAS install on that NAS! I performed the web update just like I did with the first NAS but when it rebooted and started applying the new update it said it failed and when I rebooted the USB drive I booted off had no bootstrap. Even installing back to 9.2.18 failed!
Well this was last Thursday on March 26th and so it took me until the next Wednesday, today April 1 to get it done since I had moved all my data carefully over to my other NAS during the days leading up to the upgrade attempt I was not rushed. I also had to take care of other things around the house like my yard. What I thought would be an hour update at the most turned into quite a bit more.
I started getting CPU errors plus some SCSI errors when my system tried to access my USB devices as well. The fact that I received USB errors was a very serious issue since I don’t have any internal DVD drives on these NAS systems but install off of USB thumb drives and USB DVD drives all gave errors when accessed. I could install FreeNAS from a SATA drive with the system covers open but I had to install to my USB drive for booting so that was still a no go. I figured I could update my motherboard bios and that may take care of the CPU errors but just in case I ordered a matched pair of Xeon E5345 Quad Core Processors so that when I was ready to work on the NAS they would be there in case the others were actually bad.
Well come today when I finally had some time to tackle this NAS issue I was able to get it going after installing the new Xeon E5345 CPU’s and updating the bios of my Intel S5000psl motherboard with “Intel’s Deployment Assistant” booting off of a temporarily connected internal SATA DVDRW drive I connected with the system chassis opened up. Turns out that I had to install the update since the old motherboard bios couldn’t handle the new CPU’s stepping code as well as taking care of the issue with the USB devices throwing errors. I had noted the stepping code of the new CPU’s and checked on the release notes and the new CPU stepping codes were included in the motherboard bios update.
With the new CPU’s and the motherboard updates the 9.3 FreeNAS update on my second NAS worked beautifully as I thought it would last week! I had good feelings right on initial boot off of the FreeNAS 9.3 image since when it had failed it would hang at the new FreeNAS grub screen and go no further. While following the error messages I had noticed the SCSI errors when the USB device “da0” was accessed and this no longer happened after the bios motherboard update.
I make a habit of saving my FreeNAS configuration even when doing minor updates so once I got my second NAS back into the rack and cabled up@ 9.3 FreeNAS version I logged into the web interface using the DHCP address it picked up and uploaded my old configuration and FreeNAS updated the old settings rebooting a couple of times and everything was back to normal. I verified from my XenServer shell I had the NIC’s cabled right by pinging the interfaces and then watching the storage become available in XenServer environment.
Now both NAS are @ FreeNAS 9.3 and I have my redundancy back! I also believe my old CPU’s are good but I had to test the new CPU’s to give EBay feedback plus didn’t want to switch back and forth so will repurpose the old CPU’s somehow.
Here are my babies at the bottom of my rack purring like normal. I will grab a couple of slim DVDRW devices for them for fall back devices for issues like my USB flaking on me though.
