I recently did a short 45 minutes talk at the #VirtualExpo event over at https://xenappblog.com/, about Bot management and Application firewall.
It was a short introduction to Bot management, with some live demo and then some tips and tricks for App Firewall.
If you didn’t catch it, the recording is now live, and I’ve uploaded my part here.
I wanted to create a blog post that could help the community, to use the App Firewall. Thit is one of the features that can help secure applications, running behind the ADC. But there seems to be a fear towards it, because admins have tried to enable it and something broke in the application behind it.
I think the biggest issue with App Firewall is understanding the application we want to protect, and one could argue that the ADC admin isn’t responsible for knowing how the application works.
In my experience the ADC admin is often the person that must figure out how the application works, because a lot of application owners only know how to use the application, not how it works.
So hopefully by the end of this series, you the reader, will have a better understanding on web attacks and how to counter them.
I will use an insecure web application called webgoat, that comes with lessons builtin allowing the reader to explore common vulnerabilities.
My most important advice will be to get to know the application developer. They should know how their application functions, what are expected requests and what should be blocked.
I run most of my web testing environment on a virtual Ubuntu server on my Intel Nuc.
The Ubuntu server is running a docker environment, which is perfect as webgoat got a docker image, that makes it easy to deploy.
If you want to run the webgoat/wolf using docker run, you can use the following cmd:
docker run -d -p 8080:8080 -p 9090:9090 -e TZ=Europe/Amsterdam webgoat/goatandwolf
But I like convert the run command into a yaml file, so I can use docker-compose. There is this little website that can create a yaml configuration from the run command, visit https://composerize.com/
I’ve changed the frontend port for the webgoat to port 8000, as I have something else running on port 8080.
mbp@mbp-virtual-machine:~/docker$ cat goat-with-reverseproxy.yaml
version: ‘3.3’
services:
goatandwolf:
ports:
– ‘8000:8080’
– ‘9090:9090’
environment:
– TZ=Europe/Amsterdam
image: webgoat/goatandwolf
Run it using docker-compose -f goat-with-reverseproxy.yaml up -d or add it to your docker-compose.yml
I’m running a Citrix ADC VPX 3000 Premium on my Intel nuc (long live partner licenses)
I’ve created 2 non-addressable load balancing servers on my ADC, pointing to the Ubuntu server on IP 192.168.0.10 and the ports 8000 for webgoat and 9090 for Webwolf. Then I created a couple of content switching policies, where I limit the traffic to only be accessable from my LAN and using the correct hostname.
There is also a responder policy bound on each LB to let the client know that requests against / should be to /WebGoat/ or /WebWolf/ depending on which LB the request landed at
The CLI configuration is as following:
#Server add server Ubuntu 192.168.0.10
#ServiceGroup add serviceGroup LB_SG_Webgoat HTTP -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport YES -cltTimeout 180 -svrTimeout 360 -CKA NO -TCPB NO -CMP NO add serviceGroup LB_SG_WebWolf HTTP -maxClient 0 -maxReq 0 -cip DISABLED -usip NO -useproxyport YES -cltTimeout 180 -svrTimeout 360 -CKA NO -TCPB NO -CMP NO
#Binding Servers to ServiceGroup bind serviceGroup LB_SG_Webgoat Ubuntu 8000 bind serviceGroup LB_SG_WebWolf Ubuntu 9090
#Load balancer add lb vserver LB_WebGoat HTTP 0.0.0.0 0 -persistenceType NONE -cltTimeout 180 add lb vserver LB_WebWolf HTTP 0.0.0.0 0 -persistenceType NONE -cltTimeout 180
#Binding ServiceGroup to LB bind lb vserver LB_WebGoat LB_SG_Webgoat bind lb vserver LB_WebWolf LB_SG_WebWolf
#Responder Action add responder action res_act_webgoat redirect "\"/WebGoat/\"" -responseStatusCode 302 add responder action res_act_webwolf redirect "\"/WebWolf/\"" -responseStatusCode 302
#Responder Policy add responder policy res_pol_webgoat "HTTP.REQ.URL.EQ(\"/\")" res_act_webgoat add responder policy res_pol_webwolf "HTTP.REQ.URL.EQ(\"/\")" res_act_webwolf
#Binding Responder Policy to LB bind lb vserver LB_WebGoat -policyName res_pol_webgoat -priority 100 -gotoPriorityExpression END -type REQUEST bind lb vserver LB_WebWolf -policyName res_pol_webwolf -priority 100 -gotoPriorityExpression END -type REQUEST
#Content Switch Action add cs action cs_act_Webgoat -targetLBVserver LB_WebGoat add cs action cs_act_Webwolf -targetLBVserver LB_WebWolf
#Content Switch Policy add cs policy cs_pol_Webgoat -rule "http.REQ.HOSTNAME.SET_TEXT_MODE(IGNORECASE).EQ(\"webgoat.netscaler.dk\") && CLIENT.IP.SRC.IN_SUBNET(192.168.0.0/24)" -action cs_act_Webgoat add cs policy cs_pol_Webwolf -rule "http.REQ.HOSTNAME.SET_TEXT_MODE(IGNORECASE).EQ(\"webwolf.netscaler.dk\") && CLIENT.IP.SRC.IN_SUBNET(192.168.0.0/24)" -action cs_act_Webwolf
#Binding CS policy to Content Switch bind cs vserver gw.netscaler.dk -policyName cs_pol_Webgoat -priority 200 bind cs vserver gw.netscaler.dk -policyName cs_pol_Webwolf -priority 210
WebGoat is a deliberately insecure application that allows interested developers just like you to test vulnerabilities commonly found in Java-based applications, that use common and popular open source components.
Now, while we in no way condone causing intentional harm to any animal, goat or otherwise, we think learning everything you can about security vulnerabilities, is essential to understanding just what happens when even a small bit of unintended cod,e gets into your applications.
What better way to do that than with your very own scapegoat?
Feel free to do what you will with him. Hack, poke, prod and if it makes you feel better, scare him until your heart’s content. Go ahead and hack the goat. We promise he likes it.
If you want to learn more about Webgoat, please visit https://owasp.org/www-project-webgoat/ and remember this application is made to be hacked, so having external access to it, isn’t a good idea.
When you access webgoat for the first time, you must register a new user, so click the register new user.
Make sure the username is in lowercase and accept the terms and conditions.
Now you can log on using the newly created user and start to explore webgoat. There is a lot of good information in the general area.
Let´s just jump directly into the first lession. It’s about injection and the focus for this first blog post, in the series.
There is of course a xkcd about injections.
Credit: https://xkcd.com/327/
The first test is learning how to extract information from a database, that our web application is using.
They want us to retrieve the department of the employee Bob Franco, which can be done using a Select query, like SELECT department FROM employees WHERE first_name=’Bob’
The statement can be translated to: please tell me what department users with the first name Bob, in the table employees, is a member of.
Our application got a nice form where we can submit our query, but it could a create user form, any input or query directly in the URL. How often have you seen something like ?id=5 in the URL? You could try to append your statement directly to the URL
Our statement returns the “Marketing” department. 
Great success, our insecure webapplication is insecure.
If we look with Inspect in our Chrome, we can see the query is sent.
To help against web attacks, there is a function on the ADC called Application firewall, which is a Premium licenses feature.
The App Firewall works by identifying pattern and behaviors in traffic. The simplest patterns are based on signatures. They can be found under Security – Citrix Web App firewall – Signatures
A signature is a string or a pattern that matches a known attack. The default signatures are derived from Snort (https://www.snort.org/), the ADC is able to use other lists if they are converted to XSLT files, which is the format the ADC supports.
The signatures can be updated by allowing the NSIP to contact Amazon AWS and a working DNS on the ADC. Now personally I don’t recommend allowing your NSIP to reach the internet. I would recommend you to setup a local webserver, and make it update from there.
To start with, I will create an appfw profile and policy that will log our sql injection attack. It’s a good idea to start with logging, because appfw can break web sites, that are doing something not common.
On my ADC I go the Security section and find Citrix Web app Firewall – Profiles
The profile is the collection of prevention methods we will use.
When you add the profile, you must know something about the web application, you’re going to protect.
Is a basic html application, a more web2.0 application (xml, soap) or json
For our little SQL blocking, a normal Web Application is enough (But the more ticks you make here, the more features you can enable in the profile, when we edit it)
I give it a name. I always use the naming methodology og appfw_profile_xxx (this could be the name of webapp, the function we’re protection against)
In this example it’s AppFW_prof_SQLi. Select Web application (html) and give it a comment, once you click OK, you get sent back to the previous screen. Next we have to edit it.
I select my newly created profile and edit it (I’ve put in a feature request, that we should be able to edit, when we create it)
When I open my profile, I select Security checks in the right panel, this will be the different areas we want to look at. A default policy looks like this.
I’ve removed the check marks on everything, except LOG and STATS on the HTML SQL Injection area, because I only want to focus on that part now. 
Next we have to create a policy (Remember that on the ADC there is policy that evaluates data, like a http request, response, client IP, destination port, source port and so forth) which then calls a profile that can apply an action.
I switch to the Policies – Firewall section in the Citrix Web app firewall, and add a new one.
My naming methodology here is AppFW_pol_XXX. This policy is appfw_pol_sqli, it points to my profile AppFW_prof_SQLi and the expression is true (which is an advanced policy compared to the old classic policies of ns_true), telling the ADC to always execute the profile, and I give it a comment. 
Now I have to bind it to my Load balancing server for WebGoat.
The policy is bound as an App Firewall policy on the request side.
My policy is selected. 
And the binding order is given. I select END as my goto expression, because right now I don’t have any other policies I want to evaluate.
Remember that our policy just logs and gives stats about it now, so how do we see if our policy is hit? Personally, I like to monitor the ns.log file using my ssh client.
When I log on my ADC I jump into the Shell and switch to /var/log folder, to keep on monitoring the ns.log. We can use the tail command, which default will show you the last 10 lines of a file. But I also want to keep on monitoring for any App Firewall log entry, so I use tail -f | grep APPFW, the option -f tells tail to watch for changes and update if anything is added within the last 10 lines.
Then when I try to SQL Injection on the Webgoat again, I can see it my log file.
The output is here: Feb 25 20:23:52 <local0.info> 192.168.0.240 02/25/2020:19:23:52 GMT MBP_ 0-PPE-0 : default APPFW APPFW_SQL 27820 0 : 192.168.0.3 10674-PPE0 – AppFW_prof_SQLi https://webgoat.netscaler.dk/WebGoat/SqlInjection/attack2 SQL Keyword check failed for field query=”SELECT(‘)” <not blocked>
I can see that my ADC on IP 192.168.0.240 (The NSIP) reported that client 192.168.0.3, triggered the AppFW_prof_SQLi profile with a SQL keyword check for query=”SELECT(‘)” and it wasn’t blocked, because my profile only logs and stats it.
if I want to see the same in the GUI, I can open my syslog audit message (Under System – Auditing or Security – Citrix Web App firewall – Policies – Auditing – syslog messages)
I will select the module APPFW to limit the output of the ns.log 
Now it’s time to block the SQL injection. To do so, all we have to do, is change our profile security, to block also.
When I try to enter my SQL Injection on the Webgoat, nothing happens, when I click submit. 
But I can see in my log, that the request was blocked. 
Or in the GUI.
Now a select statement might not be as dangerous you say, but they could do insert, delete or update data. Let´s look at that, while the App Firewall profile is set to LOG.
In the 3rd Injection lesson they want us to change the department of the employee called Tobi. Tobi is in Development, so maybe he created the application and therefore I don’t feel pity for him, as I change his department to sales.
My query is as following: UPDATE employees SET department=’Sales’ WHERE first_name=’Tobi’
Congratulations Tobi, you’re now in sales. Maybe Tobi should have learned how parameterized the sql statements.
And our trusted tail -f ns.log shows the following entry
Mar 1 13:37:11 <local0.info> 192.168.0.240 03/01/2020:12:37:11 GMT MBP_ 0-PPE-0 : default APPFW APPFW_SQL 107534 0 : 192.168.0.3 96299-PPE0 – AppFW_prof_SQLi https://webgoat.netscaler.dk/WebGoat/SqlInjection/attack3 SQL Keyword check failed for field query=”UPDATE(‘)” <not blocked>
Being a good ADC administrator, I noticed we had a SQL injection that wasn’t blocked. I quickly switch the profile back to block, before our friendly hacker feel pity for Tobi and want to move him back to Development department with the following query: UPDATE employees SET department=’Development’ WHERE first_name=’Tobi’
Sadly, it doesn’t work, and Tobi is stuck in Sales forever.
Mar 1 13:56:43 <local0.info> 192.168.0.240 03/01/2020:12:56:43 GMT MBP_ 0-PPE-0 : default APPFW APPFW_SQL 107967 0 : 192.168.0.3 97181-PPE0 – AppFW_prof_SQLi https://webgoat.netscaler.dk/WebGoat/SqlInjection/attack3 SQL Keyword check failed for field query=”UPDATE(‘)” <blocked>
Now let’s keep in mind that I’m a major newbie to the whole azure, so there might be smarter way of doing some of the stuff I’m going to show, but it works.
In my Azure tenant for Netscaler.dk I’ve added my work account from edgemo.com, so my validation will happen with my edgemo account.
In the Azure portal go to Azure Active Directory.
Click on Enterprise Application, where we can have a look at my “Citrix Apps” application, which is just a SAML Single sign-on application (It’s a Non-gallery Application, so make sure to select that when you add our new application).
Since I’ve already created my App, I will just click on the MyCitrixApps, to get the properties.
The properties of the Application allows you to do different stuff, like controlling Single sign-on and Conditional Access, and who gets the use the new Application. Since I want to allow single sign-on from my Citrix Unified Gateway, I have to setup the SAML information.
The information that I filled out was:
Identifier (Entity ID): https://gw.netscaler.dk
Reply URL (Assertion Consumer Service URL): https://gw.netscaler.dk/cgi/samlauth
Notice that there is an App Federation Metadata URL, which will make the setup of the SAML server on the Citrix ADC much easier. This I will show you later on.
Now to use setup Conditional Access we have to setup a new Policy, so in your App go to Security – Conditional Access – Click New.
Next up is the conditional Access. Since this is just a Proof of Concept, my conditional access will be very simple.
If the User Mads logs on to the environment using Windows, he will be required to provide MFA. If he doesn´t use Windows, a password will be enough. Now let’s look at how that is configured.
Under Conditional Access click New Policy.
I name my policy Netscaler.
Next I will select the User/group that it will be applied too.
Next I will have to setup the conditions that will apply, in my case it will be limited to Windows.
After you selected the conditions that are suited in your environment, you have to do the Access Control. For this setup it will Grant Access, but require MFA.
Lastly remember to enable the Policy and Create it.
Remember to assign your new application to a group/user
Now I won´t go into most of the Citrix ADC configuration, since most of it will be like my last blog post, which you can read here: https://netscaler.dk/?p=67
But I will cover the feature in Citrix ADC 12.1 build 51.30 which is
Metadata reading and generation support for SAML SP and IdP configuration
Citrix ADC appliance now supports metadata files as means of configuration entities for both SAML Service Provider (SP) and Identity Provider (IdP). The metadata file is a structured XML file that describes the configuration of an entity. The metadata files for SP and IdP are separate. Based on deployment, and at times, one SP or IdP entity can have multiple metadata files.
As an administrator, you can export and import (SAML SP and IdP) metadata files on Citrix ADC.
For more information, see https://docs.citrix.com/en-us/Citrix ADC/12-1/aaa-tm/saml-authentication.html#metadata-reading-and-generation-support-for-saml-sp-and-idp-configuration
This will allow us to create a SAML server, just by pointing to the XML file. If you remember when we created the Azure Enterprise application, there was a field called “App Federation Metadata URL” that contained an URL.
Keep in mind that your Citrix ADC will have to be able to browse the URL. If it only has internal Access, you can download the XML file and put in on a server that the ADC can reach.
Now to setup a new SAML policy on the ADC, go to Security – AAA Appication Traffic – Policies – Authentication – Basic Policies – SAML – Servers and click Add.
Notice the new Export and Import. We will use the Import here, since we need the iDP information.
Like always give it a name that makes sense. I normally use the following format auth_location_type, so in this case it will be auth_azure_saml.
You will get the URL for the federation metadata URL under the Single Sign-on in the Azure App.
This is the URL you will copy to the Citrix ADC. My configured SAML server looks like this.
Under More I’ve changed the Signature Algorithm to RSA-SHA256 and Digest Method to SHA256.
Next we have to make the Advanced Policy (So the workspace App can trigger validation) and bind it to the AAA server we’re using. This was all explained in the previous blog post.
Now to try something new, I’ve recorded a video. If this is something I should continue with, please let me know.
Like I started this blog post with, I’m using my work account from Azure in my private Azure Tenant. This is why my user is redirected to a local ADFS from edgemo
This is how the Workspace App looks like.
Type in the URL/Email.
Enter the username.
Same as in the video, I’m redirected to our company ADFS, because I’m using my edgemo account.
And the conditional Access prompts to for MFA.
For a smoother user experience, you can click Yes (Ja) on the following screen, to reduce the amounts of times you must re-autenticate.
Now let’s try the same on a Mac, which is my primary machine.
I enter https://gw.netscaler.dk and get redirected at once and I can select my account.
I’m redirected to the ADFS server of edgemo.
After I enter my password, I get the same question “if I want to reduce the number of logins”.
And since I’m running on a mac, I get direct access to the unified gateway, without running MFA.
Let’s take a look on the workspace app on mac. I start by adding my account.
Notice that the “browser” doesn’t remember my account, so I have to enter my email.
And I get redirected to the ADFS, where I can enter my password.
Like the last time, I get the prompt to reduce the number of logins (I keep clicking no, because I want to be able to show the whole process on request without issues).
And I get access to my applications.
And this is how my launched desktop looks like.
To get the information about the FAS, you can go to powershell. First we load the Snap-in.
Add-PSSnapin Citrix.Authentication.FederatedAuthenticationService.V1
Now we can use a command to get the certificates issues by the FAS server.
Get-FasUserCertificate -Address Citrix.netscaler.local
