December 12, 2023
by dhoytt
Comments Off on Bounced Site for XCP-ng VM Host Server Updates

Bounced Site for XCP-ng VM Host Server Updates

I had a plethora of patches to update on my XCP-ng systems and I ended up having to restart the toolstacks on my systems. I started on my slave server restarting my “XCP-ng toolstack” which caused issues as it could not contact my master in the pool due to them talking to different versions of the software packages.

Took me a while to see this was the issue because problems started when I updated the Rocky Linux 9.x system and rebooted it and it would not restart. The Rocky Linux 9.x system was of course running on my slave XCP-ng server. After a while I recalled I needed to restart the XCP-ng hosts server’s toolstack which doesn’t interrupt the VM servers running on it as that was the likely issue . When I restarted the toolstack then suddenly the slave wouldn’t come out of maintenance mode. Tried several toolstack restarts and restarting the XAPI service without luck.

I looked at some documentation and saw I needed to disable HA (High Availability) before restarting toolstack and that didn’t help. I decided to reboot but move over my VMS to the master but couldn’t due to the master being a lower version after my tootstak restart. I finally rebooted and had the same issues and took down some other VM’s I couldn’t control but were still up while the server was in maintenance mode.

After rebooting the slave XCP-ng remained in maintenance mode but with my VMs down of course. I had my ShoutCast and IceCast servers for https://dhoytt.com/snake-ice-radio-blog/now-playing/ on my XCP-ng slave host so no one was able to listen while I was doing this however my web site was till up as that VM system was on the master.

After finally looking through logs I saw that the slave was trying to communicate with the master since they were in the same resource pool. The logs showed the reason the slave would not fully boot was it could not communicate with the master. The networking was fine so I figured it was the version causing them to no communicate.

This is when I decided to gracefully shutdown the web server and other VM serves on the XCP-ng master just in case the server needed to be rebooted and not the toolstack just restarted.

Well the simple XCP-ng master toolstack restart allowed the slave to communicate to it and then come back up. The lesson I need to remember is to do all updates and restarts on the master first or switch over the slave to the master if I do it backwards which is too much unnecessary work.

Back and running with a lot of maintenance and updates needed in my server environment as I have been working tirelessly on getting the rental back up to speed and about 4 other projects that consumed my time this year.

June 29, 2023
by dhoytt
Comments Off on Rental Pool Prepped and Painted

Rental Pool Prepped and Painted

Finally finished painting the pool after lots of prep work after years of neglect in my rental. With Covid restrictions gone, I was finally able to move forward clear debris, prep surface (lots of hours) and paint. I used Dura Seal epoxy with an 8-year life. 

I had debris and sludge built up from the tenants and some campers who trespassed and dumped in the pool, very nasty.  

After cleaning the sludge, I had to pump the water out several times with my trash pump after the atmospheric rains this season. 

I then washed the entire pool down with muriatic acid. After finding different waste hazard places to dump the acid. I washed the pool with TSP. 

Due to the years of tenant abuse, there was still scaling. I tried sanding with an orbital sander and that barely “scratched the surface “. 😉 Then tried sandblasting and that was going to take way too much material and cost. 

I finally found a BAUER 9 Amp Surface Conditioning Tool surface preparation tool from Bauer and sanded the entire pool removing that hard white scale!  I took several hours through a couple of weeks after work and weekends to complete this. 

I then wanted to etch with something other than acid again prior to painting and tried several chemicals like CitrisStrip but that left a residue I had to clean up with acetone and the surface prep sander again. I finally acetone etched the whole pool and did another round of sanding with the surface prep tool.  

I had to wait a while after taping the pool off for painting. I did light washing with the hose pipe and not pressure wash to get surface dirt after doing some patching of cracks and areas when someone dropped a heavy object in the empty pool. I then waited an extra day and painted! 

Finally, I will get a chance to get water in the pool and test out the pump. I hope they did not damage the pump or that equipment! It was working fine when they moved in! 

No more amateur time, the pool will have a professional pool service maintain it and that will be written in the lease!  

Pictures from start to finish as of today. 

Tenants kept my pool like this for years and during Covid thanks to the no evictions nothing I could do and they were already behind.
After cleaning debris after campers dumped trash and I started pumping the pool.

Pumping out water process. That leaves debris to be scooped up. Some of this is what some campers from the neighborhood left.

A little guy dropped into the pool. I had to set a long 2×4 for him to eventually crawl out.

After acetone etching and some more sanding!

June 19, 2023
by dhoytt
Comments Off on Another Update Windows Music Server

Another Update Windows Music Server

I’m still running on my backup server for music stream relays and I need to update the Windows 2019 Datacenter OS with security updates.

I stepped away from doing the updates of the Fedora Linux relay servers for a few hours since its running so good on the backup system. The relay music server I took from Fedora 34 to Fedora 35 and now I’m taking it to the latest Fedora version, Fedora 38. The update system of Fedora Linux has turned out to be very reliable the last several years.

I have a backup Windows Music server but the system reboots quickly enough I just prefer to wait for the main music server to come back up. That takes less time after simple OS security updates than to sync the second server and start where I left off.

So about 15 minutes from this post time and date I will be rebooting the music server and the stream will be down for 5-10 minutes. The music relay server will be running still on the backup Linux Fedora 38 server with no music.

June 19, 2023
by dhoytt
Comments Off on Updating Relay Server Switching to Backup Brief Music Stream Interruptions

Updating Relay Server Switching to Backup Brief Music Stream Interruptions

I am updating music relay server up several versions of OS from Fedora 34 to Fedora 38 and moving the streams over to a backup VM\server already at Fedora 38. I have to make sure certificates are in place for SSL change streams from music server to this server and some items in the router and get stunnel sync’d.

Let’s see how smooth this goes. The update will be multi-stage update I have performed on the backup and other servers already.

May 6, 2023
by dhoytt
Comments Off on Brief Music Stream Interruption Updating W2019 OS

Brief Music Stream Interruption Updating W2019 OS

Needed to update Windows Server 2019 where my music resides and plays SAM Broadcaster to the latest OS security patches. The Linux systems the IceCast and ShoutCast music relays reside on didn’t get updated at this time.

Back up and streaming as I post this!

May 3, 2023
by dhoytt
Comments Off on Updated SSL Certificates Web Site and Music Streams

Updated SSL Certificates Web Site and Music Streams

My SSL site certificates expired on 05/01/2023 and 0n 05/02/2023 Tuesday I updated the website certificates. I however forgot to update the music relay certificates I run through stunnel I discovered when trying to play the network stream in secure fashion on VLC. Now the music streams for IceCast and ShoutCast can be played securely from my website of if you media player requires that secure streams.

Here’s a list of my streams and bitrates you can place in you media players or click on and it may play if you have a compatible media player or browser:

You can also look on my Now Playing Page https://dhoytt.com/snake-ice-radio-blog/now-playing/ for the following media players to listen. There may be a slight delay before they start playing of several seconds especially for the IceCast relay media players embedded on the site.

Listen 320kbps ShoutCast Listen 192kbps ShoutCast Listen 128kbps ShoutCast Listen 64kbps ShoutCast Listen 320kbps IceCast Listen 160kbps IceCast Listen 128kbps IceCast Listen 64kbps IceCast

April 13, 2023
by dhoytt
Comments Off on Getting Backyard and Garden in Shape

Getting Backyard and Garden in Shape

After all the storms, getting the fences back up plus other projects still in flight getting the backyard and vegetable garden in shape.

March 19, 2023
by dhoytt
Comments Off on Fishing Poles Racked Spatchcock Chicken Dinner

Fishing Poles Racked Spatchcock Chicken Dinner

Was able to get most of my fishing rods off the garage floor. The large surf rods I ended up breaking them down in half. I also got my most used rods closer to the ground. I still have some work to do to get my stragglers plus my telescopic rods I’ll just hang off hooks.

Now getting ready for a late Sunday evening dinner with a spatchcock chicken. I did a brine with orange zest, rosemary, apple juice, salt and pepper. Then after washing the brine off today seasoned with butter, garlic, paprika, salt and pepper under the skin and on the skin.

February 3, 2023
by dhoytt
Comments Off on Music Streams Down Temporarily

Music Streams Down Temporarily

I chose to keep the music streams down while I perform some system maintenance after some infrastructure issues I had earlier this week due to a power fluctuation. Took me a while to come up with a viable workaround while I determined the actual issues. Being under the weather with little sleep earlier this week also delayed the troubleshooting.

I’m currently moving my relay server to a new storage repository after the one it was on was corrupted shared from my FreeNAS system. Strangely enough the relay server seemed to be the only VM not previously affected by the corruption but after a reboot of the storage with a new disk and then bringing it up it had some filesystem corruption. I decided I will migrate it to a temporary new storage repository in my XCP-ng and then bring it up and fix the filesystem.

Will be back streaming soon and have the full story of this crash and the many steps to get back up and how the subsequent in motion reconfiguration and making the infrastructure more resilient at https://dhoytt.com/mainblog/.