Darryl Hoytt's Personal Blog

August 23, 2015
by dhoytt
0 comments

Updated Blog Software Friday Revisited XenServer Repair

I updated the WordPress blog software after I took some manual backups of the database. Everything went fine but of course I had to go in and manually put back in some customization’s I have made for the sites that the theme updates erased but that went smoothly.

I spread out my virtual machines more evenly in my XenServer environment and reset the HA settings on my XenServer pool. I had a couple of months ago replaced a motherboard that had failed on one of my XenServers and decided to double the RAM from a system I wasn’t using. That system kept randomly restarting as I increased the load. Finally earlier this week I ran memory check from memtest.org and received a flood of errors deep into the test (about 60% in).

I took the memory of the same speed but a different manufacture out of my XenServer system I had repaired then reran the memory test and everything was fine. This memory I took out ran the memory test fine in the other system as well but together the two memory modules from different manufactures didn’t perform together well. Had this went well I would have updated the other system to this memory level as well.

Now my environment is purring along and seeming very stable.

July 24, 2015
by dhoytt
1 Comment

Time For New Router ASUS RT-N66U Power Switch Failure

I have been having sporadic Internet connectivity issues lately and today noticed this happened several times Finally about 30 minutes ago I get texted from monitoring I have on my site notifying me my site is down, I discover I cannot contact my router and it’s literally powered off. I look over the router and the power switch is no longer depressed and will not stay depressed to keep the power on!

I had to use duct tape since electrical tape wasn’t quite adhesive enough and some folded paper to keep the power switch depressed with electrical tape keeping the edges down and smooth.

Then went on Internet and found this is a consistent issue with all ASUS routers. I have already had my share of issues from the “kernel: VFS: file-max Limit 22684 reached” error which causes me to have to occasionally reboot since that simple kernel parameter cannot be changed in ASUS’ firmware. Yes I know I can use “dd-wrt” or “tomatoe” open source router firmware but I have the VPN working nicely with this router.

Besides this recent power button issue and the kernel parameter issue I’m also not a huge fan of their interface, specifically the “Virtual Server/Port Forwarding” interface is clunky and a time sync! When I clone or switch web servers I like to just point to the new server in my environment. With ASUS I have to totally delete each prior entry and add a totally new entry for that port forwarding to the new system! Every other router interface you just change a few alphanumeric characters and apply and your done.

Time to get a new router GRRR!!!!

July 11, 2015
by dhoytt
0 comments

Moved Site to New System and OS

Well I moved my website over to a new OS Centos 7 from Fedora 21. Centos seems to be handing my enviornment better with the XFS filesystem than Fedora has been with the ext4 filesystem. I am also still making changes to the infrastructure and updated some features on the “House of Beats” (http://dhoytt.com/snake-ice-radio-blog/now-playing/) blog on this site.

I even gave the system a bit more RAM.

June 13, 2015
by dhoytt
0 comments

Home Tea From Garden Mint Stevia & Hot Sun

Can’t wait to taste this home made concoction made from some mint plants running amok in the garden and some stevia plants I added for a bit of sweet. I didn’t place a lot of stevia in the mix but will add more later to individual cup of tea to sweeten it up a bit.

This is the tea brewing in the sun with some mint in the background I let grow a bit unchecked:

This is the tea done steeping in the sun all day (note tea basket with mint & stevia on bottom of container):

This shows a couple of patches of mint in my garden that is full of mint:

This shows a couple of patches of stevia in my garden:

May 30, 2015
by dhoytt
0 comments

Work On Site This past Week

As I was migrating data from one NAS to another this week my XenServer pool master had a memory failure while I was away at work. Well after isolating the DIMM with the issue I also had to go in and change the bios settings for the fans. I noticed that the IPMI log files had numerous references to the fans and noted not all internal fans were set at optimal server fan settings.

Once I had the XenServer back in the rack cabled up it I could not see it boot up and had to switch it to another port on the KVM to see it boot and make subsequent updates. I’ll have to test the KVM port later but I suspect all is okay since other ports on that KVM are fine and it’s connected with my other KVM that also works fine. The other issue that manifested itself was that the XenServer that had the memory issue could no longer see storage from the NAS system I was moving the data off of.

I ended up doing the following to get my storage seen on my XenServer host that had the memory error:

Switched XenServer pool masters
- xe host-list <— Get UUID of =desired new XenServer pool master
- xe pool-designate-new-master host-uuid=<UUID-XenServer-host-new-pool-master>
Created a new iSCSI volume in FreeNAS.
Created new SR in XenServer pool.
Performed Full Copy of critical VM’s to other NAS storage through XenServer
Exported critical VM’s to workstation in OVF format for another bbackup
Brought up site on copy of VM on desired NAS storage
Updated FreeNAS NASto new version.
Detached storage from XenServer that had bad PBD plugins to the NAS I was moving data off for update.
Added detached storage successfully back into XenServer

I still want to reconfigure my NAS storage to make it more efficient and still have a lot of work to be done still configuring my applications that serve up my website. I just performed OS updates of the VM I have the site on and noticed Xen library updates which I hope helps in Fedora. I notice Centos behaves a little better in XenServer environment so I may switch this site to a Centos VM.

Ahh well time for a late breakfast!

May 7, 2015
by dhoytt
0 comments

Recovered Web Site and Disk Array

This week since Sunday evening I have been recovering my web site step by step and one of my NAS systems as well. This is why I have redundancy of systems and cloned systems. Still I really didn’t have my backups running the way I wanted or I would have been up right away.

I moved a few systems over each evening to my other NAS and made backups of databases and restored and copied files and made snapshots at every step so I would not have to start over. I was actually running my website on the cloned system so I revived the original system restored the MySQL backups from about a week ago and most everything was back. I still have a few plugins to install for the website and make a few tweaks. The music server being separate was up the whole time so music was still streaming.

As far as repairing the NAS the boot flash drive became corrupted. As long as I didn’t reboot it should have been fine and was since most of the executables were in memory at that point. I saved the configuration to another system, moved the virtual systems over to storage on my other NAS and reinstalled to another flash drive. I uploaded the saved configuration and everything looks great!

April 1, 2015
by dhoytt
0 comments

Updated Second NAS to FreeNAS 9.3

Well I finally updated my second NAS to FreeNAS 9.3 and it was not straightforward even though it has the exact same hardware build as the first NAS it crashed hard when I attempted the update and totally corrupted my FreeNAS install on that NAS! I performed the web update just like I did with the first NAS but when it rebooted and started applying the new update it said it failed and when I rebooted the USB drive I booted off had no bootstrap. Even installing back to 9.2.18 failed!

Well this was last Thursday on March 26th and so it took me until the next Wednesday, today April 1 to get it done since I had moved all my data carefully over to my other NAS during the days leading up to the upgrade attempt I was not rushed. I also had to take care of other things around the house like my yard. What I thought would be an hour update at the most turned into quite a bit more.

I started getting CPU errors plus some SCSI errors when my system tried to access my USB devices as well. The fact that I received USB errors was a very serious issue since I don’t have any internal DVD drives on these NAS systems but install off of USB thumb drives and USB DVD drives all gave errors when accessed. I could install FreeNAS from a SATA drive with the system covers open but I had to install to my USB drive for booting so that was still a no go. I figured I could update my motherboard bios and that may take care of the CPU errors but just in case I ordered a matched pair of Xeon E5345 Quad Core Processors so that when I was ready to work on the NAS they would be there in case the others were actually bad.

Well come today when I finally had some time to tackle this NAS issue I was able to get it going after installing the new Xeon E5345 CPU’s and updating the bios of my Intel S5000psl motherboard with “Intel’s Deployment Assistant” booting off of a temporarily connected internal SATA DVDRW drive I connected with the system chassis opened up. Turns out that I had to install the update since the old motherboard bios couldn’t handle the new CPU’s stepping code as well as taking care of the issue with the USB devices throwing errors. I had noted the stepping code of the new CPU’s and checked on the release notes and the new CPU stepping codes were included in the motherboard bios update.

With the new CPU’s and the motherboard updates the 9.3 FreeNAS update on my second NAS worked beautifully as I thought it would last week! I had good feelings right on initial boot off of the FreeNAS 9.3 image since when it had failed it would hang at the new FreeNAS grub screen and go no further. While following the error messages I had noticed the SCSI errors when the USB device “da0” was accessed and this no longer happened after the bios motherboard update.

I make a habit of saving my FreeNAS configuration even when doing minor updates so once I got my second NAS back into the rack and cabled up@ 9.3 FreeNAS version I logged into the web interface using the DHCP address it picked up and uploaded my old configuration and FreeNAS updated the old settings rebooting a couple of times and everything was back to normal. I verified from my XenServer shell I had the NIC’s cabled right by pinging the interfaces and then watching the storage become available in XenServer environment.

Now both NAS are @ FreeNAS 9.3 and I have my redundancy back! I also believe my old CPU’s are good but I had to test the new CPU’s to give EBay feedback plus didn’t want to switch back and forth so will repurpose the old CPU’s somehow.

Here are my babies at the bottom of my rack purring like normal. I will grab a couple of slim DVDRW devices for them for fall back devices for issues like my USB flaking on me though.

March 31, 2015
by dhoytt
0 comments

Backyard in Shape for Spring & Tackled Front Yard

In the backyard I am pretty much finished with spring cleanup and initial planting and this past weekend was dedicated to cleanup in the backyard, the front yard I did some trimming and planted a few more flowers and put down some black bark.

In the backyard over the winter dirt and debris that catches in the pores of the aggregate concrete so I performed my annual pressure washing of the walkways. Every spring since 1997 (even at previous residence) I setup the garden haul in whatever dirt,mulch, compost then get the rototiller out and shape everything into furrows, sweep top dirt off walkways, pressure wash the rest of the dirt and then clean the pool of any dirt and debris that gets in there. It’s amazing how much dirt you track just on your feet going back and forth getting tools, going to the store getting beverages to drink etc..

I also planted some more vegetables and placed some liquid fertilizer from Miracle Grow on my drip system so I can get some auto fertilization action going for my backyard garden. In the front yard I sprinkled some rose 2-1 one fertilizer that takes care of the bugs that gather on the roses and feeds the roses as well. I planted some marigolds and petunias as well in the front and laid out some black bark that really offsets everything nicely. The black bark which I layered in generously should keep the moisture in longer as well during the drought conditions.

I also pulled out the hammocks and they are ready for spring, summer and fall duty! Took nice dip Sunday evening in the pool and ready to roll for spring and summer!

Front Yard Flower Beds Ready For Spring with Black Bark!

Backyard Vegetable Gardens Set to go! *Note Power Washed Concrete*

Hammocks on both sides of backyard out of garage and ready for relaxation!

*Note Power Washed Concrete*

Finally pool ready for embrace!

March 22, 2015
by dhoytt
0 comments

Started 2015 Vegetable Garden Officially

The vegetable garden is officially kicked off now that I have turned over the soil with the rototiller which is always my official demarcation point for the start of my veggie garden! I was able to get all my vegetables that had maintained over the winter and replanted them this weekend except the mint plants. The mint plants grow so well and run so much I don’t think I will have any issues having a few replanted mint plants take off once back in the ground!

I replanted my collard greens, mustard greens, Swiss chard, and chives that had grown over the winter. I also ended up thinning out the chives and the Swiss chard which I should have done last year as well but didn’t.

I planted some new jalapeno peppers, green & red bell peppers, red leaf mustard greens, pear tomatoes, early bird tomatoes, beef eater tomatoes, Serrano peppers and that’s it so far. The grape vines are still hanging in there with the raspberry bushes ready to take over fertile ground if I don’t keep them in check. This year I may try to run the grape & raspberry vines over a trellis if I have time.

Below are this years before and after photos of the vegetable garden. I still have some clean up and more planting.

Look at all of the mint which overtook my garden.

These after pics were taken a bit after the sun had gone down which is when I finished. The replanted plants will perk up once watered and in the ground a few days.

March 9, 2015
by dhoytt
0 comments

Making Tea From Mints Over Running Garden

Every year this time of the year I wage a battle against mint plants over running my garden. This year I have pledged to make better use of everything in my garden including these mint pants that will not go away. Before I would include them in some alcoholic beverages an that’s it Well this year I have started making tea with them and the tea is very tasty with a dash of sugar or better yet local honey. I even think its helping me with my allergies but we will wait for that definitive declaration!

Here is the runaway plants now that never die off over the winter they just don’t spread over night like they do in the warmer months like now You can also see below that my swiss chard, mustard and collard greens did very well over the winter. I will soon uproot and replant these plants after I turn the soil with a rototiller and start incorporating other plants into the garden this year