Documente Academic
Documente Profesional
Documente Cultură
The vCenter, NSX Manager and ESXi host are functioning as the 3 main components in this architecture.
DFW Architecture
NSX Manager: The NSX manager provides the single point of configuration and the REST API entry-points in a vSphere environment
for NSX. The consumption of NSX can be driven directly via the NSX manager UI. In a vSphere environment this is available via the
vSphere Web UI itself. Typically end-users tie in the network virtualization to their cloud management platform for deploying applications.
vCenter: VMware vCenter Server provides a centralized platform for managing your VMware vSphere environments so you can
automate and deliver a virtual infrastructure with confidence.
ESXi host: VMware ESXi is the hypervisor running the virtual machines guest OS. DFW related modules:
1.
2.
vShiled-Statefull-Firewal: Service demon Runs constantly on the ESXi host and performs multiple tasks:
1. Interact with NSX Manager to retrieve DFW policy rules.
2. Gather DFW statistics information and send them to the NSX Manager.
3. Send audit logs information to the NSX Manager.
4. Receive configuration from NSX manager to create (or delete) DLR Control VM, create (or delete) ESG.
5. Part of the host preparation process SSL related tasks from NSX manager
Message Bus Client: The NSX Manager communicates with the ESXi host using a secure protocol called AMQP.
Advanced Message Queuing Protocol (AMQP) is an open standard application layer protocol for message-oriented middleware. The
defining features of AMQP are message orientation, queuing, routing (including point-to-point and publish-and-subscribe), reliability and
security
Source: http://en.wikipedia.org/wiki/Advanced_Message_Queuing_Protocol.
RabbitMQ is the NSX AMQP implementation.
The vShiled-Statefull-Firewal is acting as a RabbitMQ Client in the ESXi. The vShiled-Statefull-Firewal is a user space
service daemon and uses a TCP/5671 connection to the RabbitMQ server in the NSXmanager. The message bus is
used by the NSX Manager to send various informations to the ESXi hosts:
Policy rules for the DFW module, controller nodes IP addresses, private key and host certificate to authenticate the
communication between host and controller and requests to create/delete DLRinstances.
vSIP: VMware Internetworking Service Insertion Platform. This is the distributed
firewall kernelspace module core component. The vSIP receives firewall rules from NSX manager (through vShiledStatefull-Firewal) and downloads them down to each VM VMware-sfw.
Note: VMware Internetworking Service-Insertion Platform is also a framework that provides the ability to dynamically
introduce 3 party and VMwares own virtual as well as physical security and networking services into VMware virtual
network.
VPXA: A vCenter agent, installed on the ESXi host when the vCenter communicates with the ESXi host for first time.
rd
With the VPXA the vCenter manage the ESXi host for vSphere related tasks.Although it is not a direct part of the DFW architecture the VPXA is
being used to report the VM IP address with VMtools.
IOChains: VMware have a reserved IOchains handle packet process at the Kernel level.
Slot 0: DVFilter (Distributed Virtual Filter):
Distributed Virtual Filter DVFilteris is the VMkernel between the protected vNIC at SLOT 0 associated Distributed
Virtual Switch (DVS) port, and is instantiated when a virtual machine with a protected virtual NIC gets created. It
monitors the incoming and outgoing traffic on the protected virtual NIC and performs stateless filtering.
Slot 1: sw-sec (Switch Security): sw-sec module learns VMs IP and MAC address. sw-secis critical component capture
DHCP Ack and ARP broadcast message and forward this info as unicast to NSX Controller to perform the ARP suppression feature. swsec is the layer where NSX IP spoofgurd is implemented,
Slot-2: VMware-sfw: This is the place where DFW firewall rules are stored and enforced,VMware-sfw contains rules table and
connections table.
In figure shows below we create a firewall rule to allow ICMP Source Compute Cluster A toDestination Compute Cluster B:
The NSX Manager will need to figure out what are the object IDs represented by Compute Cluster A and Compute Cluster B and then
reveal what the IP address correspond to those VMs.
The NSX firewall rules inside ESXi host are created as VSIPIOCTL format and then applied on theVMware-sfw.
The NSX manager relies on the vCenter internal database to get object-ID/IP address mapping, we can view the data
with the vCenter MOB (Managed Object Browser) Using this url:https://vCenter_IP/mob/
The NSX manager keeps this info inside his internal database.
The vCenter server represents any object with a unique id. For example Compute Cluster A in fact equals to domain-c25 and Compute Cluster
B equal to domain-c26. Here is screenshots from vCenter MOB:
To view the vm-id we can be retrieve using vCenter at the following path:
https://<vCenter server>/mob and select content -> rootFolder -> childEntity -> vmFolder.
To view the VMs IP address go to click on the vm-id from the list above, for example vm-36 (web-sv-01a).
GuestInfo -> Net
In vCenter MOB web-sv-01a has Object ID: vm-36 with IP address 172.16.10.11.
If the VMTools was stopped or removed the vCenter removes the IP address entry immediately. An updatenotification will send to NSX manager cause
to firewall module send a list updates to all the vShiled-Statefull-Firewal processes using protobuf format. If we configure firewall rules using vCenter
objects (not IP address) as show in screenshot below, there will be a match on the last firewall rule (most of the time called catch-all rule).
If this rule configure to block (as in this example) then this VM will be blocked from the network, but if this rules send the permit then the VM
gets a full network access, which allows the user to bypass security policy.
Spoofguard: NSX feature that we can use to eliminate the need of VMtools to learn the VM IP address but this will be address in a different blog post.
When the vMotion process completes the VM1 will land at esxcomp-01b and have same firewall rules and same connection
table and as a result theres no traffic disruption for VM1.
Note: The NSX Manager is not involved in this vMotion since we dont use the Applied To feature (Explained later).
When we start using the Applied To field, NSX Manager will map the Applied To object to the
corresponding vSphere cluster. Only ESXi hosts in the cluster will receive this rule.
Each ESXi host that receives this rule will use vShiled-Statefull-Firewal demon to parse it and figure out which VMs
need to apply it. When using the Applied To field, the perimeter scope limit is thevSphere Cluster.
The shows below Rule ID 1002 in which weve configured Applied To Distributed Firewall. (Default behavior)
When we will push this firewall rule, NSX manager will send this rule to all vSphere clusters.
As a result: all VMs will get rule id 1002 at their vNic level.
Continue our example we add another rules ID 1005 in which we use the Applied To on web-sv-01aand web-sv-01b. Rule ID 1002 stay the
same with Applied To Distributed Firewall.
Assuming we have the following machines: web-sv-01a, web-sv-02a, app-sv-01A and sec-mgr-01a. And weve configured the rules above.
If web-sv-01a and app-sv-01A is part of Computer Cluster A, web-sv-02a is part of Computer Cluster B and sec-mgr-01a run in Management
Edge Cluster.
Now when we push this policy the NSX Manager need will figure rules boundaries for each cluster.
Base this rules calculation NSX manager will push the firewall update to only to Compute Cluster A and Compute Cluster B ,
Management Edge Cluster will not receive any update because is not any vSphere object part of the Applied To filed. When Compute
Cluster A received this rule firewall update vSIP kernel module will need to pars which VM will need to apply rule 1005. only web-sv-01a
will get update new rule id 1005 In addition to old rule id 1002. VM name app-sv-01a will not get any firewall rule update. When Compute
Cluster B received the rule update all ESXi host will get the firewall update inside the cluster, vSIP demon will parse it but only the ESXi
host run VM name we-sv-01b will applied rule id 1005.
Applied
To benefits:
Reduce the amount of rules per VMware-sfw, this improves efficiency because the DFW will have less rules to evaluate for every new
session.
In case of an overlap IP address within multi-tenancy environment we must use Applied To to distinguish between one tenant and others.
If the NSX Manager is down, No update rule will be push! When VM land on destination cluster no VM specific rule
apply for that VM. Its important to note that when the NSX manager is down all existing VMs forwarding plane with
DFW rules continue to work, only New VMs cannot have firewall rules until NSX Manager come back.
The NSX DFW keeps the rule table as a .dat file at the ESXi host at the following path:
/etc/vmware/vShiled-Statefull-Firewal/vsipfw_ruleset.dat .
Created in the cloud with Saaspose.Words. http://saaspose.com
NSX L2 to L4 Firewall:
The VMware NSX DFW can enforce security policy from L2 (Data Link Layer) to L4 (Transport Layer).
With L2 we can create DFW rules base on the MAC address or L2 protocol like: ARP,RARP,LLDP.
L3/L4 security rules can be enforced with a source/destination IP address or TCP/UDP ports.
Default Policy:
L3 Default Policy:
We have a default L3 policy with Source: Any, Distention: Any, Action: Allow
What youve done by implementing this rule, can be shown in the following figure:
Like the poor guy above dropping himself from his tree, by implementing this rule, you have blocked
yourself from managing your vCenter.
Resilience:
OR: How can we protect ourselves from this situation?
Put your vCenter (and other critical virtual machines) in an exclusion list.
Any VM on that list will not receive any distributed firewall rules.
Go to the Network & security tab Click on NSX Manager
Exclusion VM list 1
Double click on the IP address object. In my example it is 192.168.110.42
Click on Manage:
Go in the Exclusion List tab and click on the green plus button.
Thats it! Now your VC is excluded from any enforced firewall rules.
Exclusion VM list 6
Restoring default firewall rules set:
We can use the NSX Manager REST API to revert to the default firewall rules set to overcome amistake when we do
not yet have access to the VC.
https://addons.mozilla.org/en-US/firefox/addon/restclient
Submit a DELETE request to:
https://$nsxmgr/api/4.0/firewall/globalroot-0/config
After receiving the expected code status 204 we will revert to the default DFW policy with default
rule set to allow.
Now we can access our VC! . As we can see, we reverted to the default policy, but dont panic
saved the policy.
as we
Now we have our last policy before we blocked our VC, its loaded but not applied.
We will need to change the last Rule from Block to Allow to fix the problem.
Its not possible to disable the DFW functionality per vNIC, Exclusion List only allows to disable DFW functionality per VM.
The following list is automatically excluded from DFW functions, by default: The NSX Manager, NSX Controllers, Edge Service Gateway
and Service VM (PAN FW for instance).
Application Level Gateway (ALG) is the ability of a firewall or a NAT device that can either allow or block applications that uses dynamic
ephemeral ports to communicate. In the absence of ALG, it could be a nightmare for security and network administrators with the options of trade
off between communication and security. A network administrator can suggest opening a large number of ports which would pose security threat
for the network or the given server while a security administrator can suggest blocking all other ports except the known ports which again breaks
the communication.ALG reads the network address found inside the application payload and opening respective ports for preceding
communication and also synchronizing data across multiple sessions across different ports. For example: FTP uses different ports for session
initiation/control connection and actual data transfers. An ALG would manage any information passed on the control connection as well as data
connection in the above case.NSX-v acts as ALG for few protocols such as FTP, CIFS, ORACLE TNS, MS-RPC, SUN-RPC.
This event indicate vShiled-Statefull-Firewal demon went down on ESXi host id host-38.
We can view system event from the ESXi host itself. Here is example for FW configurationevents can be view vShiled-StatefullFirewal.log. this event cuase by policy push from theNSX manager to ESXi host. The file location is: var/log/vShiled-StatefullFirewal.log
Example for output:
2015-03-10T02:43:12Z vShiled-Statefull-Firewal: [INFO] Received vsa message of RuleSet, length 67
2015-03-10T02:43:12Z vShiled-Statefull-Firewal: [INFO] Processed vsa message RuleSet: 67
2015-03-10T02:43:12Z vShiled-Statefull-Firewal: [INFO] L2 rule optimization is enabled
2015-03-10T02:43:12Z vShiled-Statefull-Firewal: [INFO] applying firewall config to vnic list on host host-10
2015-03-10T02:43:12Z vShiled-Statefull-Firewal: [INFO] Enabling TCP strict policy for default drop rule.
2015-03-10T02:43:12Z vShiled-Statefull-Firewal: [INFO] sending event: applied vmware-sfw ruleset 1425955389291 for vnic 500e519a-87fd-4acdcee2-c97c2c6291ad.000
2015-03-10T02:43:12Z vShiled-Statefull-Firewal: [INFO] successfully saved config to file /etc/vmware/vShiled-Statefull-Firewal/vsipfw_ruleset.dat
2015-03-10T02:43:12Z vShiled-Statefull-Firewal: [INFO] Sending vsa reply of domain-c7 host host-10: 0
To view the Log filled we need to enable the Log option field.
By default when we create DFW rule there is no logging enabled. Logging occurs only afterwe enable the Log field on the firewall
rules table
In order to see Allow or Block packet in the DFW logs files we need to change the Logfield from Do not log to Log.
In the following example were changing the last rule id 1002 from Do no log to Log:
In the next example for DFW log event we will see the results of a ping from my Control VM Management IP 192.168.110.10 to
172.16.10.12:
~ # tail -f /var/log/dfwpktlogs.log | grep 192.168.110.10
2015-03-10T03:20:31.274Z INET match DROP domain-c27/1002 IN 60 PROTO 1 192.168.110.10->172.16.10.12
2015-03-10T03:20:35.794Z INET match DROP domain-c27/1002 IN 60 PROTO 1 192.168.110.10->172.16.10.12
Live Flows:
With DFW we have ability to view live flows. These flows are pulled by the vShiled-Statefull-Firewalfrom the VSIP kernel module and aggregated
appropriately. The NSX Manager pulls normal flows from vShiled-Statefull-Firewal every 5 minutes and realtime flows every 5 secs.
Enable the Flow Monitoring by clicking on Flow Monitoring -> Configuration and click on the Enable. Global Flow
Collection should change to green, Enabled status
To View the vNIC flow go to Live Flow tab and browse for a specific VM and vNIC.
Live flow will show up in the screen. The refresh Rate is 5 second.
From ESXi host the vShiled-Statefull-Firewal.log file we can see the related events:
2015-03-18T03:20:01Z vShiled-Statefull-Firewal: [INFO] Received vsa message of FlowConfiguration, length 120
2015-03-18T03:20:01Z vShiled-Statefull-Firewal: [INFO] Processed vsa message FlowConfiguration: 120
2015-03-18T03:20:01Z vShiled-Statefull-Firewal: [INFO] Loaded flow config: [120]
2015-03-18T03:21:29Z vShiled-Statefull-Firewal: [INFO] Received message in request queue of topicFlowRequest
2015-03-18T03:21:29Z vShiled-Statefull-Firewal: [INFO] Received vsa message of FlowRequest, length 52
2015-03-18T03:21:29Z vShiled-Statefull-Firewal: [INFO] Processed vsa message FlowRequest: 52
2015-03-18T03:21:29Z vShiled-Statefull-Firewal: [INFO] rmqRealTimeFlowDataRetrieve started
2015-03-18T03:21:29Z vShiled-Statefull-Firewal: [INFO] Done with configuring start of real time flows
2015-03-18T03:21:29Z vShiled-Statefull-Firewal: [INFO] rmqRealTimeFlowDataPush started
Service Composer:
Service Composer helps you to provision and assign network and security services to applications in a virtual infrastructure.
You map these services to a security group, and the services are applied to the virtual machines in the security group
Security group
With NSX DFW we have the ability to group vCenter elements such as VMs to container called security groups. Base
of this security groups we can built DFW rules. The of what vSphere object can be part of the security group can be
dynamic or static.
Security group can be consume directly in to firewall tab without use the service composer.
Dynamic inclusion:
A list of dynamic options for inclusion.
In the example below we chose VM name for any VM contains the word web:
Static Inclusion:
We can select the object type that we want to (permanently) include as part of this security group.
With static inclusion we can create nested security groups.
Static exclusion:
We can select what is the object type we want to permanently exclude as part of this security group.
We can view the security group object members from the Security Groups tab.
In our example, the Web Servers security has two VMs: web-sv-01 and web-sv-02a and this group membership is
handled dynamically because the criteria is that they have web as part of their VM name.
If a new VM, called web-sv-03a is being created in this vCenter it will automatically be part of this security group.
The service can be one L4 service or group of services, here we chose HTTPS.
We can apply this policy to any cluster running NSX Distributed Firewall.
We can apply this policy on security groups, which will result in the rule (policy) applied only to VMsthat are part of
this security groups.
vSphere environments are dynamic by nature. When new VMs join (or leave) a security group (ex.Web Servers) the vCenter will update his
database and as a result an update notification will be sentto the NSX manager. This notification will trigger a list of updates to the VMs that
are part of this security group due to being part of the Applied To. Security administrator do not need to constantly update the policy
rules for every time new VMs join or leave the network. NSX firewall policy canautomatically reflect these changes and this is the reason using
vCenter objects is so powerful
Security tag:
To keep the virtualization flexibility, without compromise security, VMware invented security tag.
By adding new tag attribute to VMs we can apply security policy. Adding or removing tag to
VM canbe done dynamically by automation, 3-Party or even manual.
We can use the example we used above in which we created a security group called Web Servers, but rather
than use a VM name containing web as the criteria for this VM group membership, we can attach a security tag to
this VM.
Create Manual Security tag:
Filter web from the list and choose web-sv-01a and web-sv-02a from list:
Now we can modify the Web servers security policy to use a dynamic including criteria:
After this changed we have two VM count with security tag web servers.
The security groups Web servers contains the VM names: web-sv-01a and web-sv-02a.
In the VMs summary page we can see the security tags that are applied to this VM and to which security group this
VM belongs. From the web-sv-01a example:
Security Policy:
Using security policy we can create templates containing DFW policy approved security admin this is how you want
to protect your environment, then apply this on security groups WHAT you want to protect. Security policy
may contain traffic redirection rules to 3rd-party vendors for service chaining.
Security policy is part of the service composer building blocks.
A different option is to apply two different security policies to same security groups.
This can result in a contradiction between the policies.
For example we apply Security Policy 1 and Security Policy 2 to WEB Security Groups:
The security policy precedency will be with a Weight value, configured by the security admin.
In the following example we demonstrate this when we create two different security policies: Allow ICMP SP and
Allow HTTP SP and apply both to the previously created security group WebServers.
Create Allow ICMP SP:
The following is the interesting part: Due to the fact that this security policy works as a template we may reuse it for
different security groups and our goal is to avoid tying this template to a specific security group.
Service: ICMP ECHO Request, ICMP ECHO Replay.
We have not applied this security policy to any security groups and so this policy has not beenactivated yet. We can see it as the gray
policy in the firewall tab.
There is no security group in the designation:
Notice that the Weight field is 1300, which is lower than the previous Allow ICMP SP 4300.
Cerate the WEB rule (same flow as above):
The firewall policy order shows Allow ICMP before Allow WEB
Now we apply both security policies on the same security group, using Apply Policy:
And do the same for the second security policy called Allow WEB SP.
In the Security Policy tab view we can see the results of this action:
From the Firewall tab we can see that now we have two activated service composer security rules.
In the service composer canvas view we have an excellent summery of the security services which were applied
We follow the same policy definition construct as DFW (i.e. same options for source field, destination field and
services field) and the only difference is in the action field: instead of Block/Allow/Reject, a user can select between
redirect/no redirect followed by a partner list (any partner that has been registered with NSX and that has been
successfully deployed on the platform).Finally: Log options can be enabled for this traffic redirection rule.
vCente
ESXi
NSX
Manage
host
VMtools
vSphere Switch
5.5 or
5.1,5.
6.0 or
vMware
Distributed
switch (vDS)
version 5.1 or
later
later
later.
VSS is not
supported
Its imported to mention that NSX DFW can work on VXLAN port-group or VLAN port-group. Enable dFW on vSS is not tested by VMware
and No supported mean if you enable it, it may work.
NSX Controller is not required with DFW. NSX Controller is only required for VXLAN and Logical Distributed Router.
The NSX DFW installation is done through the HOST PREPARATION process.
The NSX Manager triggers the NSX kernel modules installation inside a vSphere cluster and builds the NSX
Control plan fabric.
Note: Before the host preparation process we need to complete the following:
vCenter Server:
Management of vSphere compute infrastructure.
NSX Manager:
Provides the single point of configuration and REST API entry-points in a vSphere environment for NSX.
EAM (ESX Agent Management):
The middleware component between the NSX Manager and the vCenter. The EAM is part of the vCenter
and is responsible to install the VIBs (vSphere Installation Bundles), which are software packages prepared
to be installed inside an ESXi host.
The host preparation begins when we click the Install button in the vCenter GUI.
This process is done at the vSphere Cluster level and not per ESXi host.
The EAM will create an agent to track the VIBs installation process for each host. The VIBs are being
copied from the NSX Manager and cached in EAM. If the VIBs are not present in the ESXi host, the EAM
will install the VIBs (ESXi host reboot is not needed at the end of this process).
During an NSX software upgrade, the EAM also takes care of removing the installed old version of the
VIBs but an ESXi host reboot is then needed.
VIBs installed during host preparation:
esx-dvfilter-switch-security
esx-vsip
esx-vxlan
Once the host preparation was successfully completed the ESXi host has a fully working Control Plane.
Two control plan channels will be created:
RabbitMQ (RMQ) Message bus: Provides communication between the vShiled-StatefullFirewal process on the ESXi hypervisor and the NSX Manager over TCP/5671.
User World Agent (UWA) process (netcpa on the ESXi hypervisor): Establishes
TCP/1234connection over SSL communication channels to the Controller Cluster nodes.
DNS Issues:
EAM fails to deploy VIBs due to misconfigured DNS or no DNS configuration on host.
We can verify if those DFW VIBs have been successfully installed by connecting to each ESXi host in the cluster and
issuing the command esxcli software vib list.
~# esxcli software vib list | grep esx-vsip
esx-vsip
5.5.0-0.0.2318233
~ # esxcli software vib list | grep dvfilter
esx-dvfilter-switch-security 5.5.0-0.0.2318233
The message clearly indicates Agent VIB module not installed on one or more hosts.
We can check the vSphere ESX Agent Manager for errors:
vCenter home > vCenter Solutions Manager > vSphere ESX Agent Manager
On vSphere ESX Agent Manager, check the status of Agencies prefixed with _VCNS_153. If any of the
agencies has a bad status, select the agency and view its issues:
We need to check the associated log /var/log/esxupdate.log (on the ESXi host) for more details on host
preparation issues.
Log into the ESXi host in which you have the issue, run tail /var/log/esxupdate.log to view the log
From the log it appears suddenly clear that the issues may be related to DNS name resolution.
Solution:
Configure the DNS settings in the ESXi host for the NSX host preparation to succeed.
Solution:
The NSX-v has a list of ports that need to be open in order for the host preparation to succeed.
The complete list can be found in:
https://communities.vmware.com/docs/DOC-28142
If an old VIBs version exists on the ESXi host, EAM will remove the old VIBs
But the host preparation will not automatically continue.
Solution:
We will need to reboot the ESXi host to complete the process (this condition will be clearly indicated next
to the host name on vCenter).
EAM TCP/80:
If the vCenter runs on a Windows machine, other applications can be installed and alreadyusing port 80,
causing a conflict with EAM port tcp/80.
For example: By default IIS server use TCP/80
Solution:
Use a different port for EAM:
Changed the port to 80 in eam.properties in
\ProgramFiles\VMware\Infrastructure\tomcat\webapps\eam\WEB-INF\
The NSX manager has a direct link to download the VIBs as zip file:
https://$nsxmgr/bin/vdn/vibs/5.5/vxlan.zip
Reverting installation:
The control plane between the hypervisor(s) and the controllers is being down
vShiled-Statefull-Firewal is the service daemon part of UWA (User Web Agent) running on ESXi host.
To check if vShiled-Statefull-Firewal daemon is working properly, issue the following CLI
~ # ps | grep vShiled-Statefull-Firewal
36169 36169 vShiled-Statefull-Firewal
36170 36169 vShiled-Statefull-Firewal
36171 36169 vShiled-Statefull-Firewal
36172 36169 vShiled-Statefull-Firewal
36173 36169 vShiled-Statefull-Firewal
36174 36169 vShiled-Statefull-Firewal
36175 36169 vShiled-Statefull-Firewal
36176 36169 vShiled-Statefull-Firewal
36178 36169 vShiled-Statefull-Firewal
36179 36169 vShiled-Statefull-Firewal
36909 36169 vShiled-Statefull-Firewal
/usr/lib/vmware/vsfw/vShiled-Statefull-Firewal
/usr/lib/vmware/vsfw/vShiled-Statefull-Firewal
/usr/lib/vmware/vsfw/vShiled-Statefull-Firewal
/usr/lib/vmware/vsfw/vShiled-Statefull-Firewal
/usr/lib/vmware/vsfw/vShiled-Statefull-Firewal
/usr/lib/vmware/vsfw/vShiled-Statefull-Firewal
/usr/lib/vmware/vsfw/vShiled-Statefull-Firewal
/usr/lib/vmware/vsfw/vShiled-Statefull-Firewal
/usr/lib/vmware/vsfw/vShiled-Statefull-Firewal
/usr/lib/vmware/vsfw/vShiled-Statefull-Firewal
/usr/lib/vmware/vsfw/vShiled-Statefull-Firewal
The ESX shows the threads this way, these are not processes but are threads.
The vShiled-Statefull-Firewal provided activates are performed by several threads:
Firewall Rule publishing, Flow monitoring, NetX config thread, heart beat, threshold monitoring, ipfix, netcpa proxy etc Are all
supported vShiled-Statefull-Firewal activities thatare run by these threads.
Run the below command on the ESXi hosts to check for active messaging bus connection:
esxcli network ip connection list | grep 5671 (Message bus TCP connection)
Please ensure that port 5671 is opened for communication in the any external network firewall.
Logs are recorded under /var/log/vswfd.log. If the Message Bus is communicating properly with the VSM, you should see logs as follows (Heartbeats):
2015-03-10T14:10:34Z vShiled-Statefull-Firewal: [INFO] Received HeartBeart Seq 2545 , Sending response
2015-03-10T15:22:34Z vShiled-Statefull-Firewal: [INFO] Received HeartBeart Seq 2569 , Sending response
2015-03-10T16:34:34Z vShiled-Statefull-Firewal: [INFO] Received HeartBeart Seq 2593 , Sending response
2015-03-10T17:46:34Z vShiled-Statefull-Firewal: [INFO] Received HeartBeart Seq 2617 , Sending response
Since this is a module which operates at the kernel level, it is highly unlikely that the module would fail as it gets loaded as a part of the boot
image. However, in case of any failures of the distributed firewall functionality; for an instance an ESXi host maxed out on the CPU, the traffic
would be blocked by default and packets would start dropping for the VMs which are protected.You can change this default behavior to allow all
traffic to go through using an API call which would allow traffic to go through in the event of failure of distributed firewall:
You would see messaging infrastructure down on host in System Events (with host name)
New rules will not get pushed to the host. DFW UI would indicate last publish operation is pending (as opposed to succeeded true even
if its one out of 100 host that push failed).
In all cases, Enforcement of all/any of already programmed rules will Never stop.
The downtime involved will be not observable as the restart is pretty quick.
Every time a vShiled-Statefull-Firewal restart occurs, the NSX manager is contacted to sync all the rules info to make sure the
state is in sync between the NSX manager and the host.
If the vShiled-Statefull-Firewal is stopped manually on the host (i.e /etc/init.d/vShield-Stateful-Firewall stop). Then there is no attempt to
restart the process.
agentName: ESXi-Firewall
state: IOChain Attached
vmState: Detached
failurePolicy: failOpen
slowPathID: none
filter source: Invalid
port 50331655 vmk3
vNic slot 0
name: nic-0-eth4294967295-ESXi-Firewall.0
agentName: ESXi-Firewall
state: IOChain Attached
vmState: Detached
failurePolicy: failOpen
slowPathID: none
filter source: Invalid
world 35677 vmm0:web-sv-02a vcUuid:50 26 b7 4d c5 6c 1e d9-47 c0 09 25 95 80 2f ad
port 50331656 web-sv-02a.eth0
vNic slot 2
name: nic-35677-eth0-vmware-sfw.2
agentName: vmware-sfw
state: IOChain Attached
vmState: Detached
failurePolicy: failClosed
slowPathID: none
filter source: Dynamic Filter Creation
vNic slot 1
name: nic-35677-eth0-dvfilter-generic-vmware-swsec.1
agentName: dvfilter-generic-vmware-swsec
state: IOChain Attached
vmState: Detached
failurePolicy: failClosed
slowPathID: none
filter source: Alternate Opaque Channel
Slot-2: VMware-sfw: This is the place where DFW firewall rules are stored and enforced, VMware-sfw contains rules table and
connections table.
With vSphere we can capture VMs traffic with command pktcap-uw, for this example we sendcontinuous ping (ICMP echo request) packet
from web-sv-01a. The capture command will need to be place on IOChain SLOT 2 with appropriate filter name for web-sv-01a.
To find the exact filter name we need to use the command summarize-dvfilter.
We can grep the exact name with the A 3 switch mean show 3 line more after the grep term found.
From ESXi host name esxcomp-01a:
~ # summarize-dvfilter | grep web-sv-01a -A 3
world 35682 vmm0:web-sv-01a vcUuid:50 26 c7 cd b6 f3 f4 bc-e5 33 3d 4b 25 5c 62 77
port 50331656 web-sv-01a.eth0
vNic slot 2
name: nic-35682-eth0-vmware-sfw.2
agentName: vmware-sfw
From this output we can see that the filter name is nic-35682-eth0-vmware-sfw.2 for SLOT 2
pktcap-uw command help with -A output:
esxcomp-01a # pktcap-uw -A
Supported capture points:
1: Dynamic The dynamic inserted runtime capture point.
2: UplinkRcv The function that receives packets from uplink dev
3: UplinkSnd Function to Tx packets on uplink
4: Vmxnet3Tx Function in vnic backend to Tx packets from guest
5: Vmxnet3Rx Function in vnic backend to Rx packets to guest
6: PortInput Port_Input function of any given port
7: IOChain The virtual switch port iochain capture point.
8: EtherswitchDispath Function that receives packets for switch
9: EtherswitchOutput Function that sends out packets, from switch