The EMC Data Domain is a PBBA, a purpose-built backup appliance. It has one primary task and that is to serve as backup appliance and deliver benefits in your backup environment as a backup to disk target. So what are the main benefits?
Inline Deduplication: In a backup environment you backup the same data on a regular basis. With inline deduplication you save a lot of space on your appliance and even better: The more you backup the more you will save and will have a higher retention of your data.
DD Boost: Even better inline deduplication is when the application/backup software is talking to the data domain which segments are unique and only transfer the unique parts over the network. As such you are saving a lot of bandwidth and you can always perform a full backup but only the changes will be transmitted over the network. Also a lot of applications (Oracle RMAN, MS SQL, etc.) and 3rd party backup applications (Symantec, VEEAM) are supporting the DD Boost protocol.
Efficient replication: You can replicate your backups to a second location with a second data domain. Backup 2 disk and replicate it offsite … get rid of tapes and tape handling.
And when we are talking about tape handling – do you check your tapes on a regular basis? Data Domain checks it filesystem on a regular schedule and has a lot of mechanism to ensure your backuped data is valid at any time and can be restored.
Easy integration in existing environments: The Data Domain can be used as a CIFS/NFS target and can be used with almost every software. If it supports DD Boost it´s getting even better.
Deploy the OVA template of the EMC virtual Data Domain
Add an additional (third) disk to the deployed appliance for the backup data. The first two disks are system only and should not be touched or modified.
Restart the appliance
Connect via Remote Console to the data domain appliance
Login via default credentials
Start the configuration wizard
define ip settings
Connect to the Data Domain CLI via ssh
Add the third disk to the system with the following commands:
disk show hardware
storage add dev3
Connect to Data Domain via Web Interface: http://<IP/FQDN>
Set NTP time settings
Enable DD Boost
Create a user for DD Boost
User must have admin rights (needed when you add the data domain to VMWare VDP)
Create SNMP Community
The EMC Virtual Data Domain is up and running and in the next part we will integrate it to the VDP appliance. Link to the next part.
Edit 28.05.2015: The virtual Data Domain was announced during this years EMC World and is not yet available / GA.
What’s more in the Advanced version and now with vSphere 6 available for everyone:
Ability to expand current datastore
Backup to a Data Domain
Granular level restore of Microsoft Servers
Automatic backup verification
Application consistent backup support – guest-level backups and restores of Microsoft SQL Servers, Exchange Servers, and Share Point Servers, providing for application consistent backups of these servers.
8TB of deduplicated data for storage of backups (per appliance)
Up to 400 VMs per VDP Advanced appliance
No surprise, the VDP Appliance comes in the form of an OVA template. As such the first step is to deploy this template.
There is a bug right now that you can´t access the config portal via Chrome & Firefox. There is already a VMWare KB and a hotfix. But I´ve also experienced the bug with Internet Explorer and I had to install this hotfix.
Login local (via Remote Console) to the VDP
Enable root access: vi /etc/ssh/sshd_config and uncomment the line “PermitRootLogin”
Copy the hotfix to the appliance (e.g. with WinSCP)
Follow the instructions given from the VMWare KB
Login with default credentials: root / changeme
Follow the wizard to do the basic appliance configuration
Reboot the appliance
In vCenter you should see a configuration task which is finalizing the setup
After the succesful configuration task there should be a VDP menu item in the vSphere Web Client
The basic configuration settings were already set during the intial setup. But I want to show you the two configuration pages.
via VDP Web Interface: For the basic settings of the appliance. We will get back to it when we add the data domain.
via vSphere Web Client: For all Backup&Restore related tasks
The next step will be the deployment of the EMC virtual Data Domain … Link to Part 2
Unfortunately my homelab got a little bit dusty in the last few months due to a lot of work and my recent move to Munich. But with the “new” release of vSphere 6 there was the perfect opportunity to get the homelab shiny again 😉 But first of all I had to tidy up my workroom and find a nice spot for my NUCs.
ESXi 6.0 installation:
The vSphere ESXi 6.0 installation worked as usual. Just like with vSphere 5.5 you had to inject the network driver to work with Intel NUCs – but no problems at this point. You can lookup the exact procedure in one of my former posts: Click me
vCenter Server Appliance (VCSA) 6.0 installation:
With vSphere 6.0 a lot has changed with the vCenter architecture itself or let´s say got reorganized. With vSphere 5.x we were talking about vCenter Server, Inventory service and SSO. With vSphere 6.x we are talking about two “new” components:
Also the VCSA got a new deployment method – you don´t have to import a OVF/OVA anymore.
1. Install the Client Integration Plugin located within the ISO file
2. Open the vcsa-setup.html with a browser
3. Click Install or Upgrade and follow the Step-by-Step wizard
4. Here you go, up and running in a matter of minutes.
Some homelab tweaks:
1. Activate Transparent Page Sharing (TPS)
With the latest patches in vSphere 5.x and per default with vSphere 6.x there is no inter-vm TPS anymore, but only intra-vm TPS. This change was made due to some security concerns when memory pages are shared between virtual machines. There is the possibility to control which groups of virtual machines can share pages with the salt value. The salt value for the sharing group has to be the same on a per-vm setting.
In a homelab environment I don´t have any security concerns (more likely “I need more RAM!!! concerns”) and I will reactivate the classic TPS mechanism. Just set Mem.ShareForceSalting to 0 in the advanced ESXi settings.
2. Enforce small pages
A similiar discussion (already with vSphere 5.x) is with small pages vs large pages. You have to choose between more performance (large pages) and transparent page sharing (small pages). Also the VMKernel and memory mgmt will break up the large pages to small pages when the host is under memory pressure, I choose to enforce small pages from the beginning.
3. Ignore Network Redundancy Warnings
Ignore Network Redundancy Warnings:
A small tweak regarding the fact I have only one network adapter for all traffic. As such I have no redundancy at all in my homelab (shame on me! 😉 ) but I don´t want to see the annoying warning everytime.
HA Advanced Setting
Add “das.ignoreRedundantNetWarning” with value “true”
A while ago I was writing about my new pfSense firewall appliance (look here). One remaining problem was pfSense 2.1 (based on FreeBSD 8.x) was not able to operate 802.11n wifi and there was the statement the support for 802.11n will come with pfSense 2.2 (based on FreeBSD 10.x).
After some very busy first quarter in this year I´ve finally managed to upgrade my appliance to pfSense 2.2 – and I can confirm it is working with my setup. A few screens:
I have read that mixed mode (n+g) costs some performance regarding link speed, but I have to use it to connect my older devices like my tablet and I have to say that the data rate right now is enough for my purposes.
Recoverpoint for Virtual Machines will soon be available – but what is it doing?
Protects at virtual machine level
local or remote replication (sync/async)
100% software solution, no hardware appliance necessary
VMWare vCenter integrated – all administrative tasks can be done from the VMWare admin
Support for SAN/vSAN/NAS/DAS
Recover to any point in time
For those knowing EMC RecoverPoint – imagine the features of RecoverPoint as software-only solution and on a per-vm basis.
For those not knowing EMC RecoverPoint – a small picture which shows how RecoverPoint delivers Continuous Data Protection
And where does the magic happen? How does this work?
Deployment of the recoverpoint appliances & installation of RecoverPoint IO splitters to the ESXi hosts via VIB File (next part of this series)
The VMWare admin can protect a virtual machine via the WebClient through a wizard driven workflow
When an IO is issued to a virtual machine the ESXi host splits the IO with the installed RecoverPoint splitter. The original IO is directed to the storage system and the duplicated IO is redirected to the RecoverPoint appliances
Those appliances take care of replicating each IO to a Journal volume and finally to the Replica of the virtual machine disk (depending on the way [sync/async and local/remote] you are protecting your virtual machine the exact way is differing)
What did I say to a friend of mine a few weeks ago after his modem was broken: “Oh my modem is working without any concerns for years now …”
Well another 2-3 weeks later … guess what … yep my modem was also broken … karma? Long story short I was forced to get a new modem. When I was installing the new modem I was thinking about my network equipment right now and was considering to update it at another point. Behind my modem I had a pfsense firewall appliance on a ALIX2C3 board running and I was thinking about upgrading this box aswell. Therefore I was doing some research if there were any new releases of the ALIX boards … and yep there was the brand-new APU boards released.
Equipment at the moment: ALIX2C3 & pfSense
Though I have been very happy with my ALIX pfSense firewall as it was running for ages, but it had its limitations by now. (well at this point karma would have also stroke it down by lightning or by whatever anyways …)
wrong casing (the fitting casing was not available at that point)
no wi-fi because of the wrong casing (no holes for the antennas)
only 100 Mbit LAN interfaces (not bad for internet connection but between the local interfaces)
not enough CPU & RAM for additional workloads like VPN etc.
Assembling all parts wasn´t that hard but you should not forget to install the heat spreader. It´s new to the PCEngines boards that they require such a thing, but without it the board won´t live that long. Here you can find a tutorial from the manufacturer how to install it: http://www.pcengines.ch/apucool.htm
Time to boot the appliance the first time and connect via serial connection at 115200 baud. At this time we will enter the bios and adjust the boot order to boot from usb1+2 followed by the mSATA drive. We need to boot from usb to install pfSense to the mSATA SSD and afterwards we will boot directly from the SSD drive. The installation steps are explained in the following passage.
After we have written the image to the usb stick boot the appliance from the usb stick and connect again via serial connection but this time with a baud rate of 9600 (ALIX / APU bios & bootup is 115200 but pfsense 9600!). You can change the baud rate of pfSense to match the ALIX / APU board later in the settings if you want to. On the pfSense setup screen choose the third option named “Boot pfsense using USB Device” – otherwise we would boot pfsense “live” from the usb stick. But in this case we want to install it to the SSD.
I´ve installed pfSense to the SDD drive and let the installer create his recommended disk layout and partition sizes. After the straight forward installation process pull the usb stick and reboot the appliance. We will now enter the basic configuration mode on the console where we have to map the initial interfaces to its function like WLAN and LAN. We can later configure interfaces for the wi-fi and for the optional interface(s) within the web interface.
After we have configured an ip address to the LAN interface we can log in to the web interface and complete the setup wizard, which is asking for settings like name, dns servers, ntp etc. One funky new feature I´ve noticed after the final reboot is the internal pc speaker jingle sound when the pfSense bootup is completed … I was remembering long gone pc speaker games.
One still existing problem is that you can´t operate your wi-fi with 802.11 n, because of some limits within FreeBSD 8 on which pfSense 2.1 is running on. But with the new 2.2 release (not yet globally available) it will be based on FreeBSD 9 and there will be no problems with the support of 802.11n anymore.
pfSense is up and running! And if you ask why pfSense?
Well as you know I have my homelab running and like to play around with tech stuff and therefore I need a feature rich and mighty firewall appliance. Furthermore I really use some features for my daily work and now with the new board I can try out some new add ons which you can install via packages to pfSense (e.g. IDS/IPS like snort and squid as a proxy). And of course it´s a really good & secure firewall appliance I really learned to appreciate over the years. pfSense is still completely free to use but they are offering commercial support services and there are also enough shops delivering prebuilt pfSense appliances nowadays.
My top 5 things I love about pfSense:
You can set very detailed options for everything (drawback is of course it gets complex at some points)
Modular package system to install additional community driven or 3rd party software
So many features …
Update 1: 18.09.2014 13:25
John just posted a comment on this post regarding some people having boot problems with pfSense and running into an error stating “ROOT MOUNT ERROR”. I did not experience this error with any of my pfSense installations over the last years but I wanted to share a possible solution (also provided by John, thanks a lot!) to this problem for the case you are encountering such problems.
At the prompt (OK), for an SD card, type:
or for an mSATA drive, type:
After you have successfully booted the system you would have to edit the configuration file of the boot loader via the shell (/boot/loader.conf.local) to make those options persistent.
Update 2: 09.04.2015
The mentioned 802.11n wifi incompabilities are finally history with the new pfsense 2.2 release. Link
In order to get ESXi installed on the Intel NUC you have to create a customized image, because the necessary device driver for the Intel NUC network adapter (82579LM Gigabit Ethernet Controller) is not part of the ESXi base image. In order to create this image I was using the tool ESXi customizer from Andreas Peetz (www.v-front.de) in the past. The tool would still do the job (and would do it good!) but I wanted to try a new and more “onboard” way to customize the image with PowerCLI.
# Add Network Driver and check available software packages again
# Create new ESX Image and integrate software
Get-EsxImageProfile | select name
New-EsxImageProfile -CloneProfile "ESXi-5.5.0-20140302001-standard" -Name "ESXi-5.5.0-20140302001-standard-IntelNUC" -AcceptanceLevel "CommunitySupported" -vendor "GetVirtual"
# Integrate the network driver in the cloned ESXi Image Profile
Add-EsxSoftwarePackage -ImageProfile "ESXi-5.5.0-20140302001-standard-IntelNUC" -SoftwarePackage "net-e1001e"
# Export to ISO
Export-EsxImageProfile -ExportToIso -ImageProfile "ESXi-5.5.0-20140302001-standard-IntelNUC" -FilePath .\ESXi-5.5.0-20140302001-custom.iso
Installation of ESXi on the Intel NUC via VMWare Workstation:
I could now burn the image and use a USB CDROM drive to install the image to the usb stick on the Intel NUC, but I prefer a more efficient way to do the installation. I´ve created an empty virtual machine on my desktop PC with VMWare Workstation (standard vm, no disk, 2x vCPU, 4 GB RAM) and mounted the custom iso image and passed-through the usb stick.
Can´t import the usb stick because it says it is in use? Remove the drive letter, safely remove it and plug it into an usb2 port instead of usb3
Generate a new mac address when you install another ESXi with the same “ESXi helper” virtual machine to avoid some duplicate mac address in your network
Perform a standard ESXi installation and basic network configuration … I will only summarize these steps.
But there was one strange error I was getting when booting up one of the NUC boxes with ESXi stating “multiboot could not setup the video subsystem”. I´ve found a few suggestions on the net to solve this problem, but none of them fixed it anyways. The resulting problem of this error was that I couldn´t access the DCUI to make the basic configuration settings. Finally I made a work around with booting the stick in my VMWare Workstation virtual machine, made the necessary configuration settings and booted the NUC with the “pre-configured” USB stick. You can boot from a USB stick in VMWare Workstation (I´m still using version 9) with Plop Boot Manager.
Basic ESXi config
configure iSCSI connectivity
configure scratch location
Deploy W2k8 + Active Directory
deploy Windows Server 2008 R2
install AD Domain Services & DNS (Domain: get-virtual.local)
Deploy vSphere vCenter Appliance
reduce memory to 4 GB (enough reduction without editing configuration files for memory settings, VMWare KB)
basic network configuration
add AD identity source
There was an error after adding the AD identity source stating “Client is not authenticated to VMware Inventory Service”. This VMWare KB resolved the issue.
Loud (the desktop PC is louder than the HP servers …)
High power consumption
unbalanced & incompatible cluster
Future homelab components:
2 x Intel® NUC-Kit DC53427HYE with 16 GB of RAM each
HP NL54LMicroserver (G7) … will be reused as backup server
Synology DS 209+II … but there are plans to replace it with a newer and bigger box (upcoming chapter of this homelab series)
Here you can find some additional information about Intel NUCs. With the new 4th generation you have additional support for a 2.5″ drive with the existing mSATA SSD. I was also thinking about taking three of these and create a VSAN/ScaleIO cluster but finally discarded it due to the costs of the additional third box and the drives.
Summing up to about 470 € per box and I´m going to build a 2-node cluster. In my point of view that is a fair price when you look at the specs such a small box delivers (16 GB RAM!) and has the following advantages.
Assessment of future homelab:
Low power consumption
More RAM (2x 16 GB)
Less space consumption
Well it couldn´t be any easier … open the NUC and install the RAM kit … plug it in and turn it on. That´s it!
Well one thing to mention would be that the Intel NUC has only HDMI ports and no VGA/DVI resulting in diverting my television from it´s intended use. But hey … when do you get the opportunity to watch an ESXi server booting up on your large TV screen.
In the upcoming chapter I will install the basic vSphere environment (ESXi, Active Directory, vCenter Server Appliance, …).