Upgrading from CF 4.5.x to CF 5 and switching from a cluster to high availability monitoring.

If you are currently running in a CF 4.5x clustered environment, you should close all ClusterCATS explorers and reset each Web server using the ClusterCATS Server Administrator prior to upgrading to CF 5 and switching to the monitoring agent.

Start Programs ClusterCATS Server Admin Advanced Reset

During the installation of CF 5 be sure to select the correct components. Under the Application Management heading, select Monitoring, Alarms, and Hardware Load Balancing. Do not select ClusterCATS load balancing or it will override the monitoring agent and you will only have a clustering option.

After completing the upgrade and rebooting your servers, launch the CF 5 Administrator on all four Web servers:

Start Programs ColdFusion 5 Administrator

First we will set up ColdFusion probes to restart a stalled application server. At the top of the left hand panel, choose Tools. This will display the heading System Monitoring. Under System Monitoring, choose Web Servers and check the Enable box. Next choose System Probes and create a default probe by clicking the button New in the center screen. This will display probe properties; under the heading Failed Probe Response Settings, use the down arrow to choose the server you are managing: barth, brunner, pascal, or jansen. Check the blocks Restart the ColdFusion Server & Log that the ColdFusion Server has been Restarted. If you have a script that you would like to run upon server failure, check the box Execute Script and browse to the script you want executed. If you are using the dfp agent with a Local Director, check the box to Restrict Web traffic to the selected server. Server restriction with BIG-IP and other non-dfp enabled devices is not configured here, it is configured by creating a monitor or probe on the actual load balancing device and is cover later in this article. The restriction block in the CF 5 monitoring agent probe is a dfp feature which will set the server weight on CLD to 0 if the application server becomes unavailable. Click Submit Changes and the status of the probe should come up green or enabled; click back on the Web Servers link and confirm the probe status as active or enabled.

Note: You may also set up custom probes to stop and restart various services. This is a rather broad topic; it is a capability without walls deserving of many articles and case studies manifesting various configurations. You may use your imagination with this feature and feel free to share your ideas on the forums.

Note also: Make sure that the virtual directory called btauxdir is created in IIS under the IIS virtual server you are monitoring and that instead of pointing to the legacy local path c:\cfusion\brighttiger\btauxdir, it points to c:\Cfusion\cfam\btauxdir or the path to which you installed CF 5 cfam components.

Next, configure alarm notification. Click on Alarms and enter the SMTP address of your mail server. Under the heading Notification Recipients, enter your e-mail address and the address of anyone who should receive alarms from the server farm.

Under the Hardware Integration tab, there is currently only an interface for the ColdFusion dynamic feedback protocol (dfp) agent to integrate with Ciscos Local Director. If you are using Local Director in the dfp mode, then following our example you must enter hld.qa.nashua.macromedia.com beside the heading Internal DNS Name and you would enter 10.64.22.234 beside the heading Internal IP Address. For BIG-IP you would leave this blank. Look for additional options here as we further integrate with non-dfp enabled hardware load balancing devices. Note that you must enter the administrative host name and address of the CLD, not the virtual server name and address.

Configuring Cisco Local Director in dfp-mode in front of your ColdFusion 5 production Web server farm.

Step 1: Complete the CLD basic hardware installation & configuration; give the CLD an IP address & subnet mask. The IP address should correspond to a fully qualified host name (FQHN) in DNS with both forward and reverse entries. This initial IP address and name is not the one that will be visible to browsing clients; it is only a maintenance address for the CLD. In this example, the address is 10.64.22.234, and the name is hld.qa.nashua.macromedia.com. From the terminal emulator attached to the CLD type in:

enable
config t
ip address 10.64.22.234 255.255.255.0

To finish your CLD configuration using telnet from a remote client, type in the IP address and subnet mask of the remote client using the following command from the terminal emulator:

telnet 10.64.22.0 255.255.255.0

Since these settings enable you to configure the CLD remotely, you will want to write them to memory to avoid the risk of losing them:

write mem

Now that you have given your CLD an address and name, you may telnet into your CLD from the client you have specified. For this example I am using telnet and have connected to the CLD using the Fully Qualified Host Name (FQHN) hld.qa.nashua.macromedia.com; this name corresponds in DNS to the address 10.64.22.234.

Step 2: Create a virtual server in CLD. The virtual server in CLD is an IP address that must correspond to a FQHN with forward and reverse DNS entries. If you are upgrading your existing site by adding CLD into a previously clustered environment, this virtual server FQHN will correspond to what was the RR name. This is the name that will be visible to all browsing clients; it is the name placed in the client/customer browsers URL line to get to your Web site. Following our example, telnet into your CLD and add the virtual server:

hld# enable
Password:
hld# config t
hld(config)# virtual 10.64.22.235:80:0:tcp is

The above example virtual server IP address corresponds in DNS to the name www.qa.nashua.macromedia.com Step 3: Enter your real servers in CLD; real servers are what Cisco calls the servers participating in the cluster (BIG-IP refers to them as nodes). On the CLD, bind the real servers (the servers participating in the cluster) with the virtual server. This terminology may still seem strange; remember that when referring to CLD, the virtual server is the name on CLD that is visible to browsers hitting your Web site. The real servers are the ones in the CF5 server farm behind the CLD. Following our example I am adding four real servers:

hld(config)# real 10.64.22.230:80:0:tcp is
hld(config)# real 10.64.22.231:80:0:tcp is
hld(config)# real 10.64.22.232:80:0:tcp is
hld(config)# real 10.64.22.233:80:0:tcp is

Step 4: Bind the real servers to the virtual server.:

hld(config)# bind 10.64.22.235:80:0:tcp 10.64.22.230:80:0:tcp
hld(config)# bind 10.64.22.235:80:0:tcp 10.64.22.231:80:0:tcp
hld(config)# bind 10.64.22.235:80:0:tcp 10.64.22.232:80:0:tcp
hld(config)# bind 10.64.22.235:80:0:tcp 10.64.22.233:80:0:tcp

Step 5: Weight the predictor:

hld(config)# predictor 10.64.22.235:80:0:tcp weighted

Step 6: Use the CLDs dynamic-feedback command to specify the IP addresses of each real server (cluster member) and port number each server will use to listen for DFP requests from the CLD. This port number must be the same as the DFP Agent Listen Port. If you have more than one Web server hosted on a computer, even though the Web servers are in different clusters, they must have the same DFP Agent Listen Port number configured; the CF5 DFP agent will only listen on one port regardless of the number of servers. Do not use port 80. For this example I have used port 9124. If you use a port other than 9124, you will need to specify it under the Hardware Integration tab of CF 5 System Monitoring.

hld(config)# dynamic-feedback 10.64.22.230:9124 timeout 30 retry 0 attempts 10
hld(config)# dynamic-feedback 10.64.22.231:9124 timeout 30 retry 0 attempts 10
hld(config)# dynamic-feedback 10.64.22.232:9124 timeout 30 retry 0 attempts 10
hld(config)# dynamic-feedback 10.64.22.233:9124 timeout 30 retry 0 attempts 10

The DFP protocol will connect to real server 10.64.22.231 at port 9124. If the connection between the CLD and a server is closed for any reason, the CLD will attempt to reconnect, indefinitely, every 30 seconds. The CLD will close the connection if it is inactive for 60 seconds and redirect traffic to a responsive real server.

Use the CLD dynamic-feedback command settings, but do not use dynamic-feedback-pw; CF does not support secure dfp hosts. Use the dynamic-feedback-timeout option to set timeout to a value larger than the update frequency so that the CLD does not prematurely terminate the connection with the cluster because of inactivity. Macromedia Enterprise Product Engineering recommends that you set the value to at least two times the update frequency. Use the dynamic-feedback-retry option to set the retry value to zero (0) to ensure that the CLD will continue connection attempts to the CF dfp agent in the event of a lengthy period of system unavailability. For information concerning the CLD dynamic-feedback command, refer to:

Step 7. Save all these setting on CLD using the write mem command then check the configuration using the show config command:

hld(config)# write mem
Building configuration...
[OK]
hld(config)# show config
: Saved
: LocalDirector 420 Version 3.3.3
syslog output 20.7
no syslog console
hostname hld.qa.nashua.macromedia.com
no shutdown ethernet 0
no shutdown ethernet 1
no shutdown ethernet 2
interface ethernet 0 auto
interface ethernet 1 auto
interface ethernet 2 auto
mtu 0 1500
mtu 1 1500
mtu 2 1500
multiring all
no secure 0
no secure 1
no secure 2
no ping-allow 0
no ping-allow 1
no ping-allow 2
ip address 10.64.22.234 255.255.255.0
no rip passive
rip version 1
failover ip address 0.0.0.0
no failover
telnet 10.64.22.0 255.255.255.0
snmp-server enable traps
no snmp-server contact
no snmp-server location
virtual 10.64.22.235:80:0:tcp is
predictor 10.64.22.235:80:0:tcp weighted
real 10.64.22.230:80:0:tcp is
real 10.64.22.231:80:0:tcp is
real 10.64.22.232:80:0:tcp is
real 10.64.22.233:80:0:tcp is
bind 10.64.22.235:80:0:tcp 10.64.22.230:80:0:tcp
bind 10.64.22.235:80:0:tcp 10.64.22.231:80:0:tcp
bind 10.64.22.235:80:0:tcp 10.64.22.232:80:0:tcp
bind 10.64.22.235:80:0:tcp 10.64.22.233:80:0:tcp
dynamic-feedback 10.64.22.230:9124 timeout 30 retry 0 attempts 10
dynamic-feedback 10.64.22.231:9124 timeout 30 retry 0 attempts 10
dynamic-feedback 10.64.22.232:9124 timeout 30 retry 0 attempts 10
dynamic-feedback 10.64.22.233:9124 timeout 30 retry 0 attempts 10
hld(config)#

This concludes the configuration of the CLD in dfp-mode.

Configuring BIG-IP at layer two in front of your ColdFusion 5 production Web server farm.

You must attach a monitor and keyboard to your BIG-IP to start this procedure; there are some configuration variables required on BIG-IP as a layer two device that cannot be set through the browser interface. Upon booting your BIG-IP, verify that you are running 4.1. Just above the login prompt, it should read: BSDI BSD/OS 4.1 + BIG-IP Kernel. If you are running 4.0, add the PTF1 patch available for download from F-5. If you are running 3.X and you do not wish to upgrade, you will need to configure the site at layer-three instead of layer-two. The upgrade procedure to 4.0 follows:

Step 1: If you are not running 4.0, go to http://tech.f5.com/home/BIG-IP/solutions/general/sol1114.html for complete information about upgrading. Once you have downloaded the media and created a CD from the image, boot to the CDROM and ignore the warnings about your current configuration be brave! You must blow it all away to change layers. The upgrade will label the disk, format it and mount it, then reboot the BIG-IP. The upgrade will not erase your current BIG-IP software license.

reboot

Step 2. From the BIG-IP command line interface, run the BIG-IP first time boot configuration utility. The root password after the full upgrade is default.

Following our example, enter the host name of the BIG-IP:

hld.qa.nashua.macromedia.com

Press any key to continue then enter the IP address of the upstream default gateway. From our example enter:

With the next configuration screen, you have the option of setting the speed on the two interfaces. For this example I set them both to auto, but you may wish to enter the capabilities of the upstream and down stream hubs or switches.

Next, you will want to configure the external interface while leaving the internal interface un-configured. Following our example, assign the IP address to the external interface.

10.64.22.234

Next you should configure Web-browser access on the external interface. We will soon reach the point where we will be using a browser-based GUI to configure the BIG-IP. Choose the external interface and give it the same name as the host name:

Configure the names and passwords for browser access and support access. Then follow the bouncing ball by answering such basic questions as your name, address, time zone, nmt support, etc.

For the purposes of this example I have not set up BIG-IP as a DNS Proxy because we will be setting up at layer-two. While you may use this feature at layer-two, it seems intuitive that it is really more valuable at layer-three where you would want your isolated Web servers to always have access to DNS in spite of any upstream outages.

BIG-IP will ask you to confirm all your entries one at a time. Be careful to inspect all your work and make any changes as needed. BIG-IP will also generate a key for browser-based configuration.

Reboot BIG-IP. This concludes the first time boot configuration, but you may not leave the command prompt just yet. You must configure layer-two before moving to your external browser configuration tool.

Step 3. Configure the BIG-IP interfaces to work at layer-two by giving both interfaces one IP address. Currently 10.64.22.234 is assigned only to the external interface. Lets delete it from the external interface, add both the internal and external interfaces to a VLAN group called l2_mode and assign the address to the new group.

Continuing with our example, delete the self IP address using the following command:

b self 10.64.22.234 delete

Next you will want to assign both the internal and external interfaces to one VLAN group. This will allow you to assign them one IP address.

Assign the external address to the new VLAN group.

b self 10.64.22.234 vlan l2_mode netmask 255.255.254.0

Enable proxy forwarding.

b vlan l2_mode proxy_forward enable

Save your work.

Now you can ping through the BIG-IP bridge from the Web server farm, but you will not be able to ping to another subnet beyond the upstream default gateway. Reboot the BIG-IP; even though it does not ask for a reboot, you must reboot the BIG-IP at this point in order to be able to reach through any upstream default gateway beyond the subnet you are on.

reboot

Step 4. If you have not already done so, install PFT1 for 4.0 on BIG-IP. It is available at ftp.f5.com (or 216.34.94.32 if you have not set up your resolve.conf file for name resolution). Reboot BIG-IP after applying the patch.

Step 5. Now you are ready to move on to the browser based GUI to finish configuring BIG-IP. Launch a browser and following our example, you would hit the following URL:

https://hld.qa.nashua.macromedia.com

Lets confirm the current configuration. In the left side of the configuration utility screen, click on the icon Network. In the right screen you will see tabs called VLANs, VLAN Groups, Self IP Addresses, etc. Under the VLANs tab you should see external and internal; under VLAN Groups you should see l2_mode as a VLAN Group with external and internal as VLAN Groups. If you do not see this, then go back through steps 1-4.

Once you have confirmed the layer-two configuration, begin by setting up a node pool. In the left side of the configuration utility screen, click on the icon Pools. Next find the add button in the top left hand corner of the right window and add a pool. Name the pool cfservers, choose Round Robin load balancing and enter the IP addresses of your four Web servers (one at a time) under the field resources member address.

10.64.22.230
10.64.22.231
10.64.22.232
10.64.22.233

Choose http under the heading port. Ignore the member ratio & member priority fields and click on the arrows pointing to the current members field to enter each server into the cfservers pool.

Next lets select the Nodes icon in the left panel. You should now see the four nodes in your newly created pool in the center screen. Click on each IP address and under Node Properties, set a connection limit for each server. This limit will depend upon the capabilities of each server platform. I will set a connection limit of 1000 for each node. Hopefully you will use a number that reflects careful tests you ran against your application in a staging area. Macromedia Consulting Services can help you with this by providing an onsite performance and evaluation test using tools to stress your site by simulating actual production environment variables.

Next lets create a BIG-IP monitor that will probe the health of your ColdFusion application engine (in earlier BIG-IP releases, this was called Extended Content Verification or ECV). In the left side of the configuration utility screen, click on the icon for monitors. Then find the add button in the top left hand corner of the right window and add a monitor. Name the monitor cfprobe. It inherits from http. Accept the default timeouts 5 and 16. For the send string and receive rule use:

GET /btauxdir/cfprobe.cfm

Hello

Apply your changes.

Note: before setting this up, make sure you can browse to btauxdir/cfprobe.cfm and see the word Hello on each individual Web server. If you cannot get to it from your local browser, then neither will BIG-IP be able to reach it. If you cannot hit cfprobe.cfm, make sure that the virtual directory called btauxdir is created in IIS under the IIS virtual server you are monitoring and that instead of pointing to the legacy local path c:\cfusion\brighttiger\btauxdir, it points to c:\Cfusion\cfam\btauxdir or the path to which you installed CF 5 cfam components.

Now you must associate the new monitor with the servers or nodes in your new pool. Go to the Node Associations tab and choose the monitor you just created cfprobe. Use the arrow keys to select cfprobe as the Monitor Rule. Then check each box under the heading Associate Current Monitor for each node address. Apply your work.

Finally, you must create a Virtual Server on the BIG-IP and associate it with your server pool cfservers. Click on the Virtual Severs icon in the left panel of your browser screen. Following our example, enter the IP address of the virtual server. This address corresponds in DNS to the name that your browsing customer will use to hit your Web site.

Choose http or port 80. Click Next. Under the Basic Properties heading choose Enable Reset on Service Down. Under VLANS, you should see internal and external under the heading Existing. Click Next. Under the heading Select Physical Resources, choose the pool cfservers. Click Next; click Done. Go to Virtual Server Properties and set the connection limit. Since I have four servers each set to 1000, I will put 4000 in this field. You will want to set this number based on your load testing results.

Conclusion

CF 5 eliminates the need to configure a cluster of servers behind a hardware load balancing device (hld). If you running a high-end hld that sufficiently manages server load and provides content, application and transaction or server smart features (the CATS in ClusterCATS), you will want to take advantage of CF 5s capability to eliminate the overhead of clustering. On the other hand, if you have an hld that is less than optimal, and you think that clustering features may help make your site more resilient, you may wish to refer to Macromedia Knowledgebase Article 20160 for further integration information.