Kategoriarkiv: Netscaler

Modern authentication with Azure Conditional Access

Requirements

  • Microsoft Certificate Authority in Enterprise mode
  • Citrix Virtual Apps and Desktops or XenApp/XenDesktop 7.9 or newer
  • StoreFront 3.9 or newer
  • NetScaler Enterprise edition for nFactor running build 12.1 build 50+
    • The requirement is if you want to use native workspace app, if you only require browser than anything from 11.1 should be fine

Azure setup

Now let’s keep in mind that I’m a major newbie to the whole azure, so there might be smarter way of doing some of the stuff I’m going to show, but it works.

In my Azure tenant for Netscaler.dk I’ve added my work account from edgemo.com, so my validation will happen with my edgemo account.

In the Azure portal go to Azure Active Directory.

Click on Enterprise Application, where we can have a look at my “Citrix Apps” application, which is just a SAML Single sign-on application (It’s a Non-gallery Application, so make sure to select that when you add our new application).

Since I’ve already created my App, I will just click on the MyCitrixApps, to get the properties.

The properties of the Application allows you to do different stuff, like controlling Single sign-on and Conditional Access, and who gets the use the new Application. Since I want to allow single sign-on from my Citrix Unified Gateway, I have to setup the SAML information.

The information that I filled out was:

Identifier (Entity ID): https://gw.netscaler.dk

Reply URL (Assertion Consumer Service URL): https://gw.netscaler.dk/cgi/samlauth

Notice that there is an App Federation Metadata URL, which will make the setup of the SAML server on the Citrix ADC much easier. This I will show you later on.

Now to use setup Conditional Access we have to setup a new Policy, so in your App go to Security – Conditional Access – Click New.

Next up is the conditional Access. Since this is just a Proof of Concept, my conditional access will be very simple.

If the User Mads logs on to the environment using Windows, he will be required to provide MFA. If he doesn´t use Windows, a password will be enough. Now let’s look at how that is configured.

Under Conditional Access click New Policy.

I name my policy Netscaler.

Next I will select the User/group that it will be applied too.

Next I will have to setup the conditions that will apply, in my case it will be limited to Windows.

After you selected the conditions that are suited in your environment, you have to do the Access Control. For this setup it will Grant Access, but require MFA.

Lastly remember to enable the Policy and Create it.

Remember to assign your new application to a group/user

Citrix ADC Configuration

Now I won´t go into most of the Citrix ADC configuration, since most of it will be like my last blog post, which you can read here: https://netscaler.dk/?p=67

But I will cover the feature in Citrix ADC 12.1 build 51.30 which is

Metadata reading and generation support for SAML SP and IdP configuration

Citrix ADC appliance now supports metadata files as means of configuration entities for both SAML Service Provider (SP) and Identity Provider (IdP). The metadata file is a structured XML file that describes the configuration of an entity. The metadata files for SP and IdP are separate. Based on deployment, and at times, one SP or IdP entity can have multiple metadata files.

As an administrator, you can export and import (SAML SP and IdP) metadata files on Citrix ADC.

For more information, see https://docs.citrix.com/en-us/Citrix ADC/12-1/aaa-tm/saml-authentication.html#metadata-reading-and-generation-support-for-saml-sp-and-idp-configuration

This will allow us to create a SAML server, just by pointing to the XML file. If you remember when we created the Azure Enterprise application, there was a field called “App Federation Metadata URL” that contained an URL.

Keep in mind that your Citrix ADC will have to be able to browse the URL. If it only has internal Access, you can download the XML file and put in on a server that the ADC can reach.

Now to setup a new SAML policy on the ADC, go to Security – AAA Appication Traffic – Policies – Authentication – Basic Policies – SAML – Servers and click Add.

Notice the new Export and Import. We will use the Import here, since we need the iDP information.

Like always give it a name that makes sense. I normally use the following format auth_location_type, so in this case it will be auth_azure_saml.

You will get the URL for the federation metadata URL under the Single Sign-on in the Azure App.

This is the URL you will copy to the Citrix ADC. My configured SAML server looks like this.

Under More I’ve changed the Signature Algorithm to RSA-SHA256 and Digest Method to SHA256.

Next we have to make the Advanced Policy (So the workspace App can trigger validation) and bind it to the AAA server we’re using. This was all explained in the previous blog post.

User Experience

Now to try something new, I’ve recorded a video. If this is something I should continue with, please let me know.

Like I started this blog post with, I’m using my work account from Azure in my private Azure Tenant. This is why my user is redirected to a local ADFS from edgemo

This is how the Workspace App looks like.

Type in the URL/Email.

Enter the username.

Same as in the video, I’m redirected to our company ADFS, because I’m using my edgemo account.

And the conditional Access prompts to for MFA.

For a smoother user experience, you can click Yes (Ja) on the following screen, to reduce the amounts of times you must re-autenticate.

Now let’s try the same on a Mac, which is my primary machine.

I enter https://gw.netscaler.dk and get redirected at once and I can select my account.

I’m redirected to the ADFS server of edgemo.

After I enter my password, I get the same question “if I want to reduce the number of logins”.

And since I’m running on a mac, I get direct access to the unified gateway, without running MFA.

Let’s take a look on the workspace app on mac. I start by adding my account.

Notice that the “browser” doesn’t remember my account, so I have to enter my email.

And I get redirected to the ADFS, where I can enter my password.

Like the last time, I get the prompt to reduce the number of logins (I keep clicking no, because I want to be able to show the whole process on request without issues).

And I get access to my applications.

And this is how my launched desktop looks like.

How to check FAS

To get the information about the FAS, you can go to powershell. First we load the Snap-in.

Add-PSSnapin Citrix.Authentication.FederatedAuthenticationService.V1

Now we can use a command to get the certificates issues by the FAS server.

Get-FasUserCertificate -Address Citrix.netscaler.local

Citrix Workspace App and SAML/FAS

User experience

For the first setup of the workspace app, there will be a popup, where you can enter information about the environment you will connect to. Once you enter the URL/Email, it will contact the Citrix ADC if you’re from the outside, and hopefully the StoreFront directly, if you’re on the inside. But since I want to show the SAML at once, this will be a setup from external.

I’m using OKTA as my SAML provider, therefore am i redirected to Okta login

After i sign in, i will get access to my apps

I can launch my desktop

And on my FAS server I can see my certificate is created

Citrix ADC

Now to the fun part, how do we make it work.

I will mainly focus on the Citrix ADC part, as there are other great guides out there for the Citrix FAS, like the one Carl Stalhood made, which can be found here: https://www.carlstalhood.com/citrix-federated-authentication-service-saml

The Citrix ADC needs to be upgraded to firmware 12.1.50.28nc, It’s suppose to work in .49 also, but there is a kernel error which will make your Citrix ADC reboot, from what I’ve been told, by some fellow PTEC guys.

I’ve tested it both with a normal Citrix Gateway and Unified Gateway. To make it work it will require that you have an Advanced License (The old Enterprise), because we need to make use of nFactor.

On the Citrix Gateway we will have to configure an authentication Profile, so we can point our authentication traffic to an AAA server, to trigger the nFactor.

You can create the Authentication under Security – AAA Application Traffic – Autentication Profile – Add, now an authentication profile is just a pointer to the AAA server

This is how we move the authentication traffic from the Unified Gateway to AAA, where we want to do the SAML authentication.

I’ve created a SAML server for my Okta account. Like I wrote in the earlier blog post, there are some requirements for setting up SAML. We need the Redirection URL (Where will users do the authentication), the iDP signing certificate, the User Field (What will we receiver from the iDP), to sign our request from the service provider and finally the issuer name, which is the name the Citrix ADC sends to the iDP to uniquely identify the ADC.

My SAML server looks like this

Next we need a policy to call our server (remember that a policy is how we call a server/action). I’m using an Advanced Policy, which can be created under Policies – Authentication – Advanced Policies – Policy – Add

Since I want SAML to be used for everyone and everything, I will just use the policy expression true and point the action to my SAML server created earlier

Now it’s time to bind it to my AAA server, I will bind it under Advanced Authentication policies

That is basicly all you need to change, if you used a SAML policy directly on the Citrix Gateway before, which will work with browsers. But the workspace app needs nFactor to kick in.

The CLI can be found here

SAML:

add authentication samlAction auth_okta_saml -samlIdPCertName Okta -samlSigningCertName wildcard_netscaler_dk -samlRedirectUrl “https://netscaler.okta.com/app/citrixnetscalergateway_saml/secretsauce/sso/saml” -samlUserField “Name ID” -samlIssuerName “http://www.okta.com/secretsauce” -signatureAlg RSA-SHA256 -digestMethod SHA256 -logoutURL “https://netscaler.okta.com”

add authentication Policy authadv_pol_okta -rule true -action auth_okta_saml

Authentication:

add authentication authnProfile AAA-AUPL-SERVER -authnVsName AAA-SERVER -AuthenticationHost aaa.netscaler.dk -AuthenticationDomain netscaler.dk -AuthenticationLevel 3

add authentication Policy authadv_pol_okta -rule true -action auth_okta_saml

bind authentication vserver AAA-SERVER -policy authadv_pol_okta -priority 100 -gotoPriorityExpression END

Unified Gateway:

set vpn vserver UG_VPN_gw.netscaler.dk -authnProfile AAA-AUPL-SERVER

Troubleshooting:

If you have issues with authentication, we can head over to the good old aaa.debug in /tmp folder, remember this file is a pipe, so we’re looking into it as traffic is coming through.

A sample SAML authentication will a little like this:

Sun Feb 3 13:34:11 2019

/home/build/rs_121_50_16_RTM/usr.src/netscaler/aaad/naaad.c[900]: process_kernel_socket 0-64: partition id is 0

Sun Feb 3 13:34:11 2019

/home/build/rs_121_50_16_RTM/usr.src/netscaler/aaad/naaad.c[2221]: process_kernel_socket 0-64: saml_canon: preamble size is 10782

Sun Feb 3 13:34:11 2019

/home/build/rs_121_50_16_RTM/usr.src/netscaler/aaad/naaad.c[2223]: process_kernel_socket 0-64: saml_canon: read 10758 bytes from socket

Sun Feb 3 13:34:11 2019

/home/build/rs_121_50_16_RTM/usr.src/netscaler/aaad/naaad.c[2232]: process_kernel_socket 0-64: saml_canon: canon_req size is 10722

Sun Feb 3 13:34:11 2019

/home/build/rs_121_50_16_RTM/usr.src/netscaler/aaad/naaad.c[2233]: process_kernel_socket 0-64: saml_canon: input string is <saml2:Assertion xmlns:saml2=”urn:oasis:names:tc:SAML:2.0:assertion” ID=”id243710244642880201145219498″ IssueInstant=”2019-02-03T12:34:10.919Z” Version=”2.0″><saml2:Issuer Format=”urn:oasis:names:tc:SAML:2.0:nameid-format:entity” xmlns:saml2=”urn:oasis:names:tc:SAML:2.0:assertion”>http://www.okta.com/SecretSauce</saml2:Issuer><ds:Signature xmlns:ds=”http://www.w3.org/2000/09/xmldsig#”><ds:SignedInfo><ds:CanonicalizationMethod Algorithm=”http://www.w3.org/2001/10/xml-exc-c14n#”/><ds:SignatureMethod Algorithm=”http://www.w3.org/2001/04/xmldsig-more#rsa-sha256″/><ds:Reference URI=”#id243710244642880201145219498″><ds:Transforms><ds:Transform Algorithm=”http://www.w3.org/2000/09/xmldsig#enveloped-signature”/><ds:Transform Algorithm=”http://www.w3.org/2001/10/xml-exc-c14n#”/></ds:Transforms><ds:DigestMethod Algorithm=”http://www.w3.org/2001/04/xmlenc#sha256″/><ds:DigestValue>6pq1us+JQQXPIWJ8R/p+ZH7Vb1DYUNyUCZvhg4WaFoY=</ds:DigestValue></ds:Reference>

You can also use a SAML tool in the browser to decode. I like SAML Message Decoder for Chrome myself and the Developer tools in the browsers are also useful.

The flow is as following

  1. Client: GET / HTTP/1.1
  2. ADC: HTTP/1.1 302 Object Moved Location: /vpn/index.html
  3. Client: GET /vpn/index.html HTTP/1.1
  4. ADC: HTTP/1.1 302 Object Moved Location: /logon/LogonPoint/tmindex.html
  5. Client: GET /logon/LogonPoint/tmindex.html HTTP/1.1
  6. ADC: HTTP/1.1 200 OK
    1. Now the Client and ADC will talk until the Client Ask for Authentication and ADC will respond with SAML
  7. ADC: GET /nf/auth/doSaml?act=auth_okta_saml;nf=;wv=0 HTTP/1.1

Now the Client will talk with the SAML iDP and here it will do a SAMLRequest, which is a base64 encoded message (Look for a POST against the iDP which contains the SAMLRequest) as seen here:

Now this big block of data, is something you can decode online. I like https://www.samltool.com/decode.php, where you can paste the whole SAMLRequest, and it will return something like this

So there is a lot of information to troubleshoot SAML on the Netscaler and in the browser

Citrix Netscaler 12.1.50 Saml Issue

Hey,

It’s been a while since i updated my blog, but i thought it would be time to pick it up again.

So i was playing around with the native receiver (workspace app) and SAML/FAS, as i’m having some issues getting this to work, so i wanted to set it up my own little test environment at home.

For that i had to setup some SAML authentication on my Netscaler running build 12.1.50-28.nc and i kept getting an error while trying to add the SAML server

To setup a basic SAML policy we need to add the SAML iDP server which you can do under Citrix Gateway – Policies – Authentication – SAML – Servers – Add

To make the SAML server you need a couple of things

  1. Name
  2. Redirect URL
  3. Single Logout URL
  4. SAML Binding
  5. IDP Certificate Name
  6. Signing Certificate Name
  7. User field
  8. Issuer Name
  9. Signature Algorithm
  10. Digest Method

But whenever i tried to add the server i got the following message
Arguments cannot both be specified [samlIdPCertName, metadataUrl]

I have to admit there have been a lot of GUI issues in the Netscaler lately (Like the Invalid Argument AES256 from last patch) so i jumped into the CLI to see how little i should add before Netscaler would accept it and the GUI would allow me to work with it.

The CLI for adding a SAML server is something like this: add authentication samlAction auth_Okta_saml -samlIdPCertName Okta -samlSigningCertName -samlRedirectUrl “https:///adfs/ls/” -samlUserField “Name ID” -samlIssuerName

But if you don’t prefer the CLI and want to use the CLI, the least amount of configuration i could get the Netscaler to accept and allow me to edit the server in GUI was this: add authentication samlaction auth_Okta_saml -samlIDPCertName Okta -samlSigningCertName Cert -samlredirectUrl https://fqdn

After that i could edit the server and change any setting, so my guess is that the Netscaler got an issue in regards to the Redirect URL

p.s i know the Netscaler is Citrix ADC now, but i love the old name to much, sorry Citrix 🙂

How to troubleshoot network issues with the Netscaler

Hi all,

It´s been a while since my last blog post, but I have been busy at work, working on a very fun XenMobile project.

Last post I promised that I would explain how we can use the built-in tcpdump function in the Netscaler, without having to take a packet capture and open our trusted Wireshark.

I´ve been on a lot of assignments, where I had to setup something for the customer, like a SSL Offload for Outlook Web Access, and once I’ve done the service part, I quickly notice the service is in a downstate as shown in this picture.

Outlook web access service is down
Outlook web access service is down

Now we can check why the service is in a downstate by double clicking the service, and checking the monitor

Monitor down, Timeout doing TCP connection establishment stage
Monitor down, Timeout doing TCP connection establishment stage

This tells me that when the Netscaler tries to make a TCP connection to the service using the monitor HTTP, it sent the packets, but didn´t couldn´t make a proper TCP connection, something is blocking the connection. Since I almost never touch the backend service, like the OWA in this example, I would ask the contact person on site, if they were sure that the subnet IP of the Netscaler is able to communicate with the backend – hence, is there firewall blocking the communication. I often get an answer like “Yeah, we check the firewall and nothing is blocking the traffic, the problem must be on your end”

Now I make sure that I have the correct IP address, that I’m trying to connect to it on the correct port number, if there are a lot of different interface i.e. a lot of different SNIP, I make sure the Netscaler will send the traffic using the  correct SNIP.

Now a way to check that the traffic is flowing like I want it, I could fire up the packet capture on the Netscaler, download the pcap file and run it in my trusted Wireshark, this is a very effective way of debugging, but seeing as this is a simple issue, I can just use the built-in tcpdump function.

I SSH to the Netscaler, change to shell and fire up my nstcpdump.sh
The OWA backend service ran on IP 10.10.200.5, so I want to monitor the traffic flowing from the Netscaler to that IP.
In my shell cmd I enter: nstcpdump.sh dst host 10.10.200.5, this will show me traffic sent to the destination host 10.10.200.5, the output will look like this

output of nstcpdump.sh dst host 10.10.200.5
output of nstcpdump.sh dst host 10.10.200.5

We can see that 10.10.200.16 is sending a packet to 10.10.200.5, but the ack 0, this means that the Netscaler have not received a reply from destination, and the pattern is the same in the following packets.
Just a note “the first packet sent wouldn´t be able to have an ack number, since the source haven’t communicated with the destination yet.”

The output tells me the follow, the Netscaler is trying to communicate with the backend server from SNIP 10.10.200.16, it´s connecting to the backend from a random TCP number, but the destination port number is 80/http like expected. I can now go back to my contact person, saying that I can see the Netscaler is behaving as I expected.

I would say from experience that 9 out of 10 times the traffic is being blocked by a firewall.
Once we get the service in an UP state, the output of nstcpdump.sh DST host 10.10.200.5 would look like

Service is UP again
Service is UP again

It´s easy to see the difference between a down and up service using nstcpdump.sh

There are lot of other useful filters, but take a look at the CTX article located at http://support.citrix.com/article/CTX118185.

That is all for now, next time we will take a look at XenMobile App Wrapping, I just did a fun job where I had to hack an iOS application, so I could wrap it and upload it to the Citrix AppController.

Take care out there.

 

 

How to troubleshoot authentication

Hi,

I can´t count how many times I’ve been told that the Netscaler isn´t letting users log on, so no one can work.

In 99% of the cases it´s not the Netscaler that is failing, but the external authentication service we are using, so unless you work with local users on the Netscaler, then the Netscaler will ask an external authentication server to authenticate an user.

Let us have a look at what happens when an user tries to log on using an AGEE and they fail their login.

Logonfailed

The user gets the message “Incorrect user name or password” When we have to figure out what is going on, we can turn to the auditing – syslog on the Netscaler

Logonfailesyslog

 

 

(Click the picture for a larger version of it)

 

The picture tells us the AAA module had a login_failed for the user mbptest the reason is “External authentication server denied access” this is tell a Netscaler admin, that it wasn´t the Netscaler itself that denied the user access to the system. However, it doesn´t say what authentication server was asked, what the reason for deny is, so the only useful information we got, was that it wasn´t the Netscaler itself.

 

Now if we want to a much better way to figure out what is going, we can use the aaad.debug module, this module is a pipe, so nothing is saved to disk, but require we do live monitoring of it.

 

To get access to the aaad.debug we need to use the command line of the Netscaler, so we can go System – diagnostics – command line interface, which will open a console on the Netscaler from the GUI, but it´s rather limited so I much rather start up my trusted SSH client and connect to the Netscaler.

 

Once we got access to the Netscaler, we have to go into NSCLI (Netscaler Command Line Interface) so type in shell and press enter, this will change the prompt from > to the user@hostname#

 

Go the /tmp folder using cd /tmp, and try to type ls -l, you will find aaad.debug in this folder, so now we just need to monitor the file, while we do a login, and to do that we can use the command cat, you can find the manual page for cat here http://unixhelp.ed.ac.uk/CGI/man-cgi?cat

 

So to monitor the aaad.debug, we will use cat aaad.debug, now we will see everything that touches the AAA daemon, ask the user to log on again and follow the authentication.

aaad_debug_ldap_start

We can see that the user mbptest is starting an LDAP authentication against the server 10.10.10.11

aaad_debug_ldap_ssl_bind

 

The next thing that happens here is the connection to the server is using SSL/TLS, and the connection to the 10.10.10.11 is using SSL (ldaps port 636) then the bind event starts and finally the bind event is successful.

The bind is when the user we use to access the ldap server, so on our LDAP server we added a service account, that is used to access the ldap, now if the bind fails, then no one will be able to log on, because we can´t access the ldap server.

Common issues when bind fails are password expired, the account is logged out of the domain, and account is disabled.

The next that happens is a bind event for the user, where we will check the ldap for the user account, figure out what groups/nested groups the user is member of, and finally ldap will return the result of the bind event

aaad_debug_ldap_user_fail

We can see that the user is located, but the error is invalid credentials (i.e. wrong password)

Therefore, we checked that the Netscaler could communicate with the LDAP server, the service account works (the first bind is successful) but the user is typing in a wrong password.

If we have a primary and a secondary authentication server (like radius and ldap) then the auditing – syslog would still just say, “external authentication server denied access” but using aaad.debug we can check how far into the authentication the users gets.

For the next blog I will talk about using NSTCPDUMP.SH for live packet monitoring without the need for a wireshark.

How to troubleshoot policies in realtime.

Hello,

There is an quick and easy way to see what policies applied in realtime using the command line

If you havn´t had time to check out the nsconmsg command, this post will help you master it.

The command can be used with the AGGE, rewrite and responder policies, and i find that it´s the fastest way to debug what is going on.

The first thing you need to know, is that you have to be in shell mode, for it to work, so after you SSHd into the Netscaler (I prefer to use Putty http://www.chiark.greenend.org.uk/~sgtatham/putty/download.html on Windows and just use the buildin on my Mac)

So after you access to the netscaler, you have to type Shell like this:ShellAccess

After you got access to the shell, you can use nsconmsg, the parameters that i mostly use is the this:

nsconmsg -d current -g pol_hits

When an user logs on to the AGEE, it will display  which authentication policies that was used and the session policies (if the login was successful of course) an output will look like this:

nsconmsg

This picture shows what policies was hit in realtime.

There are a couple of other paramets that are helpful:

nsconmsg –d current | egrep –i rewrite/responder depending if you want check for rewrites or responder policies.

Hopefully this quick post will help Netscaler administrators to debug AGEE, rewrite and responder policies in realtime.

My next blog post will be about authentication troubleshooting in realtime also.