First a quick background for those unfamiliar with the P4000 line. HP bought a small company called Lefthand, which made a software product called SANiQ, this product could be installed on a server with lots of storage and then the box magically becomes an iSCSI SAN. Big deal you say?… Well what makes the Lefthand (now called HP P4000) so unique is its modular design in which nodes can be grouped together to form clusters for fault tolerance and increased capacity.
So basically this post is a walk through of what you need to do in order to remove a node from a cluster and how to add new nodes to the cluster. This process is important if you are:
- Retiring old nodes
- adding storage capacity
- migrating from SATA nodes to SAS nodes
- any other reason that you might want to add or remove a node 🙂
Because this procedure is fairly important in terms of getting everything right to avoid data loss or corruption I want to add an extra disclaimer in addition to the one on the about page.
This post in no way guarantees complete information, while I do my best to make sure all angles are covered… It is only meant to be a quick overview of how the procedure works. Before you do anything to equipment that could cost you your job, you should visit HP.com and get the official P4000 manuals and make sure you know the entire process, or call HP support. I take no responsibility for your actions on your hardware.
To get started I will assume that you have your SAN up and running and all new nodes have been given an IP address and are reachable from the “old” SAN nodes. Please note that this process will move data to the new nodes so you will want to make sure that you have Gigabit ethernet between all nodes for this process. Also I would make sure that your SAN is stable for this procedure, only the HP gods know where your data will be if a source or destination node goes offline during this process.
The first thing to do is to “find” your new nodes. To do this you can enter their IP addresses into the “Find Systems” wizard. After your nodes have been found they will show up in the tree on the left. At this point we have our nodes online and the other systems should be able to communicate with them.
The second step is to add the available systems to the Management Group in which they will be used. To do this right click the Management Group they need to go in (in this example we are going to add node SAS-1 and SAS-2 to the “Production” Management Group). After right clicking on the management group select “Add Systems to Management Group”. A wizard will appear and ask you to select which nodes to add, select both of the new nodes and click “continue”.
Now the nodes will move from the “Available systems” area into the “Production” Management Group. They are not however part of the “Production Cluster” yet.
Here is what it should look like when you have the nodes added to the Management Group.
The next procedures will vary depending on what exactly you are doing. You are only adding nodes to an existing group then you can skip the part where we remove the SATA-1 system from the cluster. If, however, you are replacing a node or migrating from one node(s) to new ones then you can remove them as this article shows.
Adding Nodes to the Cluster
To add nodes to the cluster right click on the cluster and select “Edit Cluster”. A box will appear with a list of nodes currently in the cluster, on this screen click “Add Systems”.
As you can see in the previous screenshot I have the SATA-1 node highlighted. If you are only adding nodes DO NOT DO THIS NEXT STEP. If you are getting rid of the existing nodes then highlight the old system(s) and click “Remove Systems”. Then click “O.K.”
What happens now?
Basically, the management software on the SAN will figure out that we are adding new nodes and in this case, it also knows that we are removing a node. So it knows to move the data off the node(s) that are getting decommissioned and re-stripe the data across the new nodes. This process will take a long time depending on the amount of data you have on your volumes. The best place to monitor progress is on the “Volumes and Snapshots” section. See the screenshot, but basically, this tab will show you the re-stripe progress of each volume and each snapshot.
As you can also see in this screenshot besides the Restriping status is the list of Storage Systems on the left. We can now see that we are using SAS-1 and SAS-2 for this cluster. and the SATA-1 node which was the only node in our cluster is now in the management group … but not listed as an available system (at least yet). After ALL volumes and snapshots are restriped we can safely shutdown the old node.
That is pretty much it, its a fairly straightforward process. I would, however, make sure all your stuff is backed up OFF OF THE SAN, and offline like a tape or separate backup server.
As a side note, I thought I should mention that even during this migration process all of you data will be online and available. The screenshot below was taken while my SATA node restriped data to the 2 SAS nodes, and during that time I was able to copy data to the Volume.