The other day I was asked to help install a new DAE on a VNXe3100 for a customer. While it is a very simple process I thought I would share what I look for to make sure that things are working properly.
Step 1: Rack and Cable
After unboxing the new shelf we first need to get it into its new home, be it a rack or where ever. Next we need to cable the shelf, this part involves 4 cables: 2 power cables and 2 SAS cables. A fully populated DAE should draw no more than 250-300 watts under a heavy load, but make sure that where ever you are plugging in the power there is ample room.
The next part of cabling the system is to plug the SAS cables in. One end will attach to the new shelf and the other end will attached to the DPE (head unit) or the shelf below it, depending on if this is the first expansion shelf or not.
Remember that there is an “A” side and a “B” side to both units. The VNXe3100 shelves are easy to tell because they are on the left and right, but the VNXe3300 shelves are more like the VNX shelves so pay attention when you connect things. For the VNXe3300 shelf the “A” fabric is the bottom link control card, and the “B” fabric is the top LCC.
The DPE will always have small diamonds by the SAS ports and the shelves have diamonds and circles. REMEMBER always start with DIAMONDS (so the DPE side) and then connect the other end of the SAS cable to the circles (Shelf side). I like to remember it by saying you have to buy your girlfriend an engagement ring before you get married (wedding band = circle).
The back of the VNXe DPE’s are different depending on the model you have, here is the VNXe3300 connected to a DAE:
(Click the image for a larger version)
Next we have the VNXe3100, which looks a little different because its a 2u model and only has 1 SAS port per controller, unlike the 3300 which has 2 sas ports per controller.
Refer to these diagrams on how to connect the new shelf to the DPE, the purple lines are the SAS cable on the “A” fabric, and the red lines are the SAS cable for the “B” fabric. After getting the shelf connected to the DPE you will see the LCD display on the back of the shelf switch from “–” on both line control cards, to a number, this number will indicate the shelf number as the DPE sees it. If the LCD displays two dashes only, then the DPE does not see the shelf properly. Once it sees the shelf properly you can move on to the next step.
Step 2: Verifying the new drives are seen
Next login to Unisphere and click on the “System” tab, next find the “System Health” button and select it. Inside of System Health you will see a list of hardware on the left side. If you can see the DAE listed there then your VNXe is able to see the new hardware. To be more thorough you can click the drop down arrow to the left of DAE and it will list out all the disks that it can see in it. Verify that it sees the correct number of disk drives.
Step 3: Putting the new drives to work
The final step that we need to do is put our drives to work, and to do that we have top options.
- Add the drives to an existing drive pool
- Create a new drive pool
Which one you select will depend on what you are trying to accomplish. You should work with your VAR, or EMC to determine which is the best use of your new hardware.
If you are going to add them to a pool select the existing pool and click details. Inside of the details window there is an “Add Disks” button at the button, click that and follow the wizard and your new drives will be added to the pool.
If you are going to create a new pool, simply follow the same steps you did when creating your old pools.
That about does it, like I said it is a very easy process and should take you no more than a few hours to complete.
Is this done live? Or is it a best pratice to shut down the vnxe first?
It can be done online. SAS allows for hot plugging, so when you add the cables to the new shelf it assigns the shelf an address and such on the fly.
Just a note the VNXe 3300 diagram shows you connecting to the SAS port 1 instead of 0. This causes an error and does not allow the DAE to show up as normal. I just spent about 40 minutes reading articles to discover that while the VNXe 3300 has 2 SAS ports only 0 is available for use. Just thought you may want to know and maybe correct the diagram.
Thanks Dennis, Ive ran into that too, didnt even think to put it on the diagram though… Ill update it asap.
Thank you for your blog. It has been really helpful. I followed your directions, 2nd shelf added, have 5 each 300GB 15k drives running on it and they show up in Unisphere as “unconfigured Disk Pool”. However, it am getting an error when trying to add additional drives to my existing pool, getting a “There are not enough disks available for this storage pool” error.
Current Perf Pool has the same disks installed (7 each 300GB 15k drives with one hot spare).
Not sure why the VNXe will not allow me to add any of the 5 new drives. Thoughts?
Thx ahead of time
Jerry, I just sent you an email. Let’s do a webex and see whats going on.
Thanks for the great blog post. Two questions on this:
First, after racking and cabling the new DAE, do you run the SAS cables before turning the new DAE on? Or turn the new DAE on and wait for it to load before plugging in the SAS cables?
Also, as Dennis said SAS 0 is what needs to be used and SAS 1 is not able to be used. Seems like I’ve ran into this before and had to get EMC involved. They were able to make it work, but can’t remember how. So they question is, if you are adding a second DAE, since port 1 on DPE is “not available” to use, do you need to run the second DAE to the first DAE?
It really doesnt matter in my opinion on whether you do power or SAS cables first.
I typically do SAS cables first then power when im ready… but again ive seen it done either way and it really doesnt matter… think of it like an ethernet switch instead of a SAS switch… you can plug ethernet cables in then power, or plug ethernet cables in after power … either way they negotiate and move on.
A question about replacing disks to VNXe. I’ve heard that there might be problems unless disks are manufactured by EMC. Is it possible to use generic disks or should should I play it safe?
Standard disks wont work. EMC uses custom firmware and disk geometries on their disks. for example a normal disk uses 512 byte sectors, EMC disks use a 520 byte sector…
Pingback: EMC's VNXe Server Architecture – A Primer | ServerMania Blog