See http://kb.lsi.com/KnowledgebaseArticle15753.aspx#vmware for a KB article on support for all 3ware controllers and VMware versions.

Driver and instructions also available directly from VMware here: http://www.vmware.com/support/vsphere4/doc/drivercd/esx-400.2.26.08.035vm40-1.0.4.html

A certified driver for the LSI 3ware 9650SE and 9690SA VMware ESX/ESXi 4.0 and ESX/ESXi 4.0 update 1 is available.  For ESXi the driver only includes secondary (e.g. non-booting) support.  Third party solutions have not been tested by LSI.

The certified driver and installation instructions are here.  See also these links:

http://www.vmware.com/support/vsphere4/doc/drivercd/esx-400.2.26.08.035vm40-1.0.4.html

http://www.vmware.com/resources/compatibility/search_results_ajax.php?action=search&deviceCategory=io&productId=1&advancedORbasic=advanced&maxDisplayRows=50&key=&release[]=-1&datePosted=-1&ioTypeId[]=-1&manufacturer[]=35&chipset[]=-1&firmware[]=-1&nports[]=-1&vid=&did=&svid=&ssid=&rorre=0&&partnerId[]=220

3ware 9650SE & 9690SA VMWare ESX/ESXi 4.0/4.0 update 1 Driver Installation Procedure.

Prerequisites:

a. 3ware 9650SE and/or 9690SA RAID Controller(s).
b. ESX and/or ESXi 4.0/4.0 update 1 Installation Media
c. 3ware driver image on CD (click here to download)
d. Windows 2000; XP; Server 2003; Vista or Server 2008 on the local Network

VMWare ESX4.0 & 4.0 update 1:

1). Install 3ware RAID Controller, drives and define RAID Unit(s). (Please reference 3ware Quick Installation Guide for further information)

2). Boot from the VMware ESX 4.0 or 4.0 update 1 DVD-ROM. When prompted for additional driver support, please provide the 3ware driver CD from here and click next. Once the driver is loaded proceed with the installation, choosing the volume, etc.

3). Download and install the VMWare vSphere Client/CLI Version 4.0.0-161974.

4). Launch the vSphere client providing the root account and password. Once the vSphere Client is running,
you can setup Vms; Storage Pools, etc.

5). You MUST run VMware patch ESX400-200907001 for ESX 4.0 or ESXi400-VEM-200907001 for ESXi 4.0
    due to a bug in the VMkernel not calling the drivers pci_driver->shutdown entrypoint, causing the array to
    occasionally get an unclean shutdown.   This issue will be fixed in ESX/ESXi 4.0 "update 1", at which time,
    the above patches are no longer needed.

Please note:  You can use the 64-bit tw_cli from the 9.5.2 code set or newer http://www.lsi.com/channel/ChannelDownloads/

VMWare ESXi4.0/4.0 update 1:
(RAID Device can only be Secondary Storage)

1). Install 3ware RAID Controller, drives and define RAID Unit(s). (Please reference 3ware Quick Installation Guide for further information)

2). Boot from the ESXi 4.0/4.0 update 1 Installation Media and install to Primary Hard Drive.

3). Once completed take note of the IP address or hostname of the ESXi server and make sure you can ping the host.

4). From the Windows system, download and install the combination VMware vSphere Client/CLI Version 4.0.0-161974.
General link: http://www.vmware.com/download/vsphere/open_source.html
VMWare CLI: http://communities.vmware.com/community/developer/vsphere_cli
VMWare client:  Please follow these steps to download the needed VMWare client:
   a.  Go to http://www.vmware.com, and click on "Downloads"
   b.  Next to VMware ESXi click on "Free Download"
   c.  Under 'VMware ESXi 4 Installable' click on "Download".  
   d.  At this point you must login.  If you don't have a login with VMware, you will need to create a free account.  
   e.  After logging in, click on "Continue".  You will now be able to download "VMware vSphere Client and Host Update Utility". 

5). Download and install the VMWare-viclient 4.0.0-162856
General link: http://www.vmware.com/download/vsphere/open_source.html
VMWare CLI: http://communities.vmware.com/community/developer/vsphere_cli
VMWare client:  Please follow these steps to download the needed VMWare client:
   a.  Go to http://www.vmware.com, and click on "Downloads"
   b.  Next to VMware ESXi click on "Free Download"
   c.  Under 'VMware ESXi 4 Installable' click on "Download".  
   d.  At this point you must login.  If you don't have a login with VMware, you will need to create a free account.  
   e.  After logging in, click on "Continue".  You will now be able to download "VMware vSphere Client and Host Update Utility". 

6). Insert the 3ware Driver CDROM or medium that contains the driver file. Copy the file, 'AMCC_2.26.08.035vm40-offline_bundle-179560.zip'
to the root of C:\ e.g. C:\AMCC_2.26.08.035vm40-offline_bundle-179560.zip.

7). Launch the vSphere Client providing the credentials for root.

8). Put the ESXi server into Maintenance Mode from Vsphere Client.
(Double click on Inventory-->Click Summary tab-->click Enter Maintenance Mode.)

9). Go to Programs>VMware> and launch the vSphere CLI.

10). You should have a command prompt showing the following path, 'C:\Program Files\VMware\VMware vSphere CLI'

11). Change directory to bin e.g. 'cd bin'

12). Now enter the following command via the VMware CLI:
perl vihostupdate.pl -server x.x.x.x -username root -password "" -b c:\AMCC_2.26.08.035vm40-offline_bundle-179560.zip -i

If you see this error: "metadata.zip.sig does not exist/signature mismatch : metadata.zip/Unable to unpack update package" be sure that you are using version 4.0 of the VMware RCLI , and not an older version.

If you see this error: "Install operation failed: SOAP request error - possibly a protocol issue: invalid value 0" be sure to check the permissions with the VMware ESXi root account.  You may need to change the credentials to another Administrator account.

13). Once the driver load using vihostupdate is finished, close the VMware CLI.

14). Go back into the vSphere Client and take the Esxi server out of maintenance mode
(Click Summary tab-->click Exit Maintenance Mode.)

15). Reboot it.
(Click Summary tab-->click Reboot, then Yes to confirm)

16). As the ESXi server comes back up, you should see the 3warexxx drier loading on the ESXi server.

17). Once the system has fully completed the reboot and all services have loaded, on Vsphere you'll see the connection to the ESXi Server will be automatically reestablished through the vSphere Client.

18). Now you are ready to create VMs; Storage Pools, etc.
vSphere

19). You MUST run VMware patch ESX400-200907001 for ESX 4.0 or ESXi400-VEM-200907001 for ESXi 4.0
    due to a bug in the VMkernel not calling the drivers pci_driver->shutdown entry point, causing the array to
    occasionally get an unclean shutdown.   This issue will be fixed in ESX/ESXi 4.0 "update 1", at which time,
    the above patches are no longer needed.

Please note:  When running VMware ESXi 4.0 with the 9650SE and/or 9690SA, you can't use the tw_cli from a standard 3ware code set.  You should use the tw_cli from here.  You can run this special version of the attached binary of tw_cli in ESXi 4.0 by ssh’ing into the machine.  The user's guide for the tw_cli gives the syntax for all commands, and can be found on the 3ware support page http://www.lsi.com/channel/ChannelDownloads/.

CLI Syntax Legend:
x.x.x.x - ESXi Server System IP

"" - default password [if not changed]