Today I installed Update 2 for ESXi 5.0 and ran into a problem that was very weird. I upgraded a few hosts using VUM and everything looked good so went ahead and updated them all. For us we are running a Nexus 1000v for our distributed switch setup. After all the hosts had been brought out of maintenance mode I looked to make sure everything was still up and running. All of a sudden ran into a problem where some of the VM's were not connected to the network. Looked at the networking tab for the DVS and noticed that the VM's that were disconnected had an X through their connections. I thought this was weird and looked through all of our hosts and realized that this was only happening on half of our hosts. Everything else looked good though it was not like the hosts themselves had lost connectivity to the 1000v or their NIC's became disconnected or anything like that, they were all still showing the proper configuration. I decided to go into the 1000v to make sure that it was seeing all the VEM modules. I looked and it saw all the VEM's but all of a sudden the hosts that were having problems were showing as unlicensed. I looked at the license distribution and all of a sudden ghost VEM's showed up in the licensing file and stole the licenses from actual hosts. I know these were ghosts because they did not show up when you did the show module vem command and if you even tried to list the actual VEM that it was showing licensed, i.e. show module vem 3, it would say VEM does not exist. So I had to manually go in and transfer the licenses from the ghost modules to the actual live modules. I still don't know why this happened but I wanted to put this out there in case anyone else runs into this type of problem, I know I could not find anything written up about this. Here is the link to the PDF file with all the licensing commands used if anyone needs it.
↧