Automatic Proxy Configuration using PAC File, Batch Script and Windows Server 2012 GPP

Introduction

The proxy auto-configuration is a technique which defines how & where the web browser and other application requests are redirected. Also, this mechanism is designed to overcome the changes and difficulties of manual configuration. Instead of using the static proxy server address, the web browser or application executes a JavaScript function for every request. This script provides greater flexibility than a manual configuration. This blog explains about automatic proxy configuration using Proxy Auto Config script (PAC), windows batch script, and Group Policy Preference (GPP).

What is a PAC file & what it does?

PAC stands for Proxy Auto Configuration. This file contains a set of rules coded using a JavaScript function FindProxyForURL (url, host) which determines whether web browser requests (HTTP, HTTPS, and FTP) go direct to the destination or forwarded via web proxy server such as squid proxy. Vmoksha uses squid proxy server, which is a fully featured web proxy cache server application.

Before Proxy Automation in Vmoksha:

We were facing following problems and challenges before automatic proxy configuration,

  • Manual Proxy Configuration
  • Speed & Latency Issues
  • No Failover Setup
  • Exception (Proxy Bypass) Configuration
  • Explicitly proxy disables in external networks
  • Internet connectivity outage

Benefits after Proxy Automation in Vmoksha:

The following are the benefits after automating proxy configuration,

  • No Manual Effort
  • Script-driven method of controlling the routing of web requests
  • Proxy bypass configuration for private sub-networks, internal/local hosts, and local domains
  • Support for all major operating systems and web browsers
  • Automatic proxy failover with multiple proxy servers
  • Efficient and automated traffic routing regardless of domain name or IP address
  • Support for wireless networks in mobile devices
  • Supports web traffic load balancing

Procedure:

The following are the steps for automatic proxy configuration

Step 1: PAC File Creation

Create a PAC file script based on the following example.

PAC File Example:

function FindProxyForURL(url, host) {

// If the hostname matches, send direct.
   if (dnsDomainIs(host, "localdomain.com") ||
       shExpMatch(host, "(*.localdomain.com)"))
       return "DIRECT";

// If the protocol or URL matches, send direct.
   if (url.substring(0, 4)=="ftp:" ||
       shExpMatch(url, "http://localdomain.com/folder/*"))
       return "DIRECT";

// If the requested website is hosted within the internal network, send direct.
   if (isPlainHostName(host) ||
       shExpMatch(host, "*.local") ||
       isInNet(dnsResolve(host), "10.0.0.0", "255.0.0.0") ||
       isInNet(dnsResolve(host), "172.16.0.0", "255.240.0.0") ||
       isInNet(dnsResolve(host), "192.168.0.0", "255.255.0.0") ||
       isInNet(dnsResolve(host), "127.0.0.0", "255.255.255.0"))
       return "DIRECT";

// DEFAULT RULE: All other traffic, use below squid proxy servers in fail-over order.
   return "PROXY 1.2.3.4:3128; PROXY 5.6.7.8:3128";
}

url – The full URL being accessed in web browser. (http:// or https:// or ftp://)

host – The hostname from the above url. port numbers and sub-location is not included in this

return – Return value can be any of the following

  • DIRECT – Redirects requests directly to the destination
  • PROXY host:port – Redirects requests to Proxy server
  • SOCKS host:port – Redirects requests to SOCKS server

Finally, save the file with .pac extension. Eg. proxy.pac

Step 2: Host the PAC file on a web server for client access

Next step is to host the PAC file on a web server’s home directory such as (/var/www/html for Apache2) or (/usr/share/nginx/html for Nginx) or (C:\inetpub\wwwroot for IIS8) and make sure the file is accessible from intranet clients.

Step 3: AutoConfigURL setting via Group Policy Preference

Context: To configure Internet Explorer with a Proxy PAC file using Group Policy Preferences options.

  • Open your GMPC.MSC console and navigate to User Configuration / Preferences / Windows Settings                 
  • Right Click on the Registry object from the left hand pane and select New > registry Item

Automatic Proxy Configuration

From New Registry Properties, login in the following settings

  • For Hive: HKEY_CURRENT_USER
  • For Key Path: Software\Microsoft\Windows\CurrentVersion\Internet Settings
  • For Value name: AutoConfigURL
  • For Value Type: REG_SZ
  • For Value data: http://mysite/proxy.pac

Screenshot

  Proxy.pac File

Apply and OK to complete this GPP Configuration

The following are steps to auto configure winhttp proxy settings

Step 1: Batch Script Creation

Create two batch scripts with the following content

Script 1: "setproxy.bat"

rem This Batch File sets the WinHTTP proxy settings and bypasses the localhost

netsh winhttp set proxy "proxy.mydomain.com:8080"; 127.0.0.1,localhost

Script 2: "resetproxy.bat"

rem This Batch File resets the WinHTTP proxy settings

netsh winhttp reset proxy

Step 2: Group Policy Object Creation

  1. Open your GMPC.MSC console and create a new Group Policy Object and enter the name of it.
  2. Navigate to User Configuration / Policies Windows Settings / Scripts
  3. Select Logon under Scripts; add the above script “setproxy.bat”
  4. Select Logoff under Scripts; add the above script “resetproxy.bat”
  5. Navigate to Computer Configuration / Policies / Administrative Templates / System / Group Policy
  6. Enable the policy Configure Logon Script Delay, and enter “0″ minute.
  7. Attach this GPO to the appropriate OU of your domain and enable it.
FacebookTwitterGoogle+Share
Setting up a Secure Email Engine using Amazon SES

Cloud computing, also known as on-the-line computing, is a kind of Internet-based computing that provides shared processing resources and data to computers and other devices on demand. It is a model for enabling ubiquitous, on-demand access to a shared pool of configurable computing resources (e.g., networks, storage, applications, servers, and services), which can be rapidly provisioned and released with minimal management effort. Cloud computing and storage solutions provide enterprises and users with various capabilities to store and process their data in third-party data centers. It relies on sharing of resources to achieve coherence and economy of scale, similar to a utility (like the electricity grid) over a network.

Amazon Web Services (AWS), a subsidiary of Amazon.com, which offers a suite of cloud computing services that make up an on-demand cloud computing platform.  The scope of this blog is confined to one of the efficient and effective services which are a part of AWS – Amazon SES.

Amazon SES is a pay-per-use email distribution engine that provides AWS users with an easy, authentic, cost-effective, reliable and consistent infrastructure for sending and receiving bulk email correspondence using your domain and email addresses. 

amazon web services

Why Vmoksha opts for Amazon SES?

Amazon SES works with Elastic Compute Cloud also known as “EC2,” Lambda, Elastic Beanstalk and various other services. It is available in different regions such as US-East, US-West, and EU-Ireland, which allow consumers close to these regions to deploy their applications to ensure high availability and low latency.

Unlike other SMTP players in the market, Amazon SES provides competitive pricing and deliverability.

Listed below are certain benefits of using Amazon SES:

  1. Trusted by Internet Service Providers (ISP) as an authentic source
  2. Cost-Effective & Competitive Pay-per-use pricing
  3. Reliability and Scalability
  4. Bulk Messaging Engine
  5. Automation using Amazon Lambda functions
  6. Ensure deliverability and Active monitoring to make sure that the illegal or questionable content is not being distributed
  7. No Infrastructure challenges
  8. Provides mailbox simulator application as a testing environment
  9. Real-time notifications via Amazon SNS.

How Vmoksha make use of Amazon SES?

The Amazon SES service along with Amazon Lambda service is configured for sending emails automatically. The mail sent via SES is verified by ISP and mail service provider such as Google and finally delivered to the employee(s). To ensure the smooth delivery of the mail, Vmoksha undergoes certain workarounds, which are described in the following sections.

The following diagram explains the scenario

Amazon SES

Amazon Simple Email Services

Setting up Amazon Simple Email Service (SES):

First, set-up Amazon Web Services (AWS) account to use this service

After signing up to the AWS account, log-in into the management console and look for SES under services section or log-in with the URL, http://aws.amazon.com/ses

 

Steps to verify Email Addresses and Domain:

   I.  Steps to Configure Amazon SES

Goto SES home page, navigate to Identity management menu and choose your option to verify either your email domain or list of addresses.

For example;

Email addresses – sales@abc.com, finance@abc.com and so on…

Domain – abc.com

The verification is managed using the Amazon SES console or Amazon SES API.

Note: Email address and domain verification status for each AWS region is separate.

Although, Email Addresses verification is quite an easy step, completed by opening the verification URL sent by SES. Domain verification demands the following steps,

    1. Go to Domains under Identity Management, select Verify a New Domain.
    2. Enter the domain name and select Generate DKIM settings and Click Verify This Domain.
    3. List of DNS record details will be displayed, which needs to be added in the DNS Zone Files of your domain. Eg. Godaddy DNS management
    4. Download the csv file of DNS Records. This contains the details of Text (TXT), Canonical Name (CNAME), and Mail Exchange (MX) records that need to be added or amended in DNS records.
    5. Domain verification can be done by just adding a text (TXT) record in your DNS Zone File. But, it is highly recommended to perform DKIM verification.
    6. TXT Records looks similar to this,

 

_amazonses.abc.com         TXT     pmBGN/7MjnfhTKUZ06Enqq1PeGUaOkw8lGhcfwefcHU=

 

  1. On propagating TXT record in domain, the domain verification status changes to verified
  2. To ensure that the mail is from a trusted source, DKIM verification is required. DKIM verification can be done by adding CNAME records in DNS Control Panel.
  3. Once DNS changes are reflected, the domain is fully verified.

Email Authentication via SPF or DKIM:

Amazon SES uses Simple Mail Transfer Protocol (SMTP) to send an email. Since SMTP does not provide authentication by itself, spammers can send messages pretending to be from the actual sender or domain. Most of the ISPs evaluate the email traffic to check if the emails are legitimate.

 

Authentication Mechanisms:

There are two authentication mechanisms used by ISPs commonly:

  1. Email Authentication with SPF (Sender Policy Framework)
  2. Email Authentication with DKIM (DomainKeys Identified Mail)

 

Email Authentication with SPF:

Setting up SPF Records and Generating SMTP credentials:

A Sender Policy Framework (SPF) Record indicates to ISPs that you have authorized Amazon SES to send mail for your domain. SPF Record looks similar to this,

abc.com       SPF           “v=spf1 include:amazonses.com -all”

 

SMTP Credentials can be generated from SES management console under Email Sending section. It prompts to create an IAM user and provides SMTP username and password upon creation of that IAM user. Another alternative way is to create a separate IAM user with access to SES service using access key and secret key as SMTP credentials.

Note:

If SPF Record already exists, then, you can append “include:amazonses.com” to the existing record. Also to work with Google apps, you need to add “include:_spf.google.com ~all”

If SPF record does not exist in the DNS Zone File, text (TXT) record can be added with the value as “v=spf1 include:amazonses.com -all.”

 

Email Authentication with DKIM:

DKIM (DomainKeys Identified Mail) is a standard that allows senders to sign their email messages & ISPs and use those signatures to verify whether that messages are legitimate and cannot be modified by a third party in transit. DKIM setup can be done by adding CNAME records provided by Amazon SES in DNS Zone File.

Here are the samples of CNAME records for DKIM Verification,

mvkw7orpsecw2._domainkey.abc.com  CNAME  mvkw7orpsecw2.dkim.amazonses.com
jp5x3nni3zf4uo6._domainkey.abc.com CNAME  jp5x3nni3zf4uo6.dkim.amazonses.com
7i3j33udxinbhjf6._domainkey.abc.com  CNAME 7i3j33udxinbhjf6.dkim.amazonses.com

 

Finally, now it’s time to leave all SMTP servers and move on to AWS Simple Email Service (SES). This way Amazon Web Services reduces the effort of DevOps and takes IT Revolution to the next level.

Useful Links:

Expose your localhost to web in 50 seconds using Ngrok

 

An introduction to Open Source Technology

The term “Open Source” in general refers to something that can be modified because its design is publicly accessible. Open source technology is defined as the production and development philosophy of allowing end users and developers to not only see the source code of software, but modify it as well. Ngrok is one such project developed by GitHub.


What is Ngrok?

Ngrok is a multiplatform tunnelling, reverse proxy software that establishes secure tunnels from a public endpoint such as internet to a locally running network service while capturing all traffic for detailed inspection and replay.

 

How does Ngrok works            Image Source: https://ngrok.com/

Why Ngrok? & How we have deployed it in Vmoksha?

Before using ngrok, when we needed to expose a localhost application to web (internet) all we were doing is deploying the application in a server running a DMZ or we used to relocate the host to DMZ and configure NATing in the firewall. We also used to make DNS configuration in External DNS where the domain is hosted. In general, DMZ (De-Militarized Zone) is a computer host or small network inserted as a “neutral zone” between a company’s private network and the outside public network. It prevents outside users from getting direct access to a server that has company data. The following are the issues that we were facing before Ngrok deployment:

  • Unable to expose localhost application directly to internet without DMZ & other network configuration
  • Unable to demonstrate an application to Client on urgent basis
  • Unable to share websites for testing purpose
  • Develop any services which consume Webhooks (HTTP CallBacks)
  • Can’t share a website temporarily that is running only on our developer machine
  • Time Consuming on network and DNS configurations
  • Can’t debug or inspect HTTP Traffic in a precise manner
  • Can’t run networked services on machines that are firewalled off from the internet
  • Unable to expose application behind http proxy
  • Unable to forward non-http and non-local network services

 

Architecture before Ngrok deployment

 

Architecture before Ngrok deployment

 

Real-time Ngrok Usage in Vmoksha (A Case Study):

After using Ngrok, we had addressed all the about requirements and mainly it serves our business need in faster, secure and easy manner.  Here is the following how we had used the features of Ngrok in our (Vmoksha) environment. In our scenario we used ngrok in Windows as follows:

As this is a small 9MB executable(.exe) tool we can be generally executed with Ngrok command followed by the port no which has to be exposed as follows,

port-8080

 

Which gives a random subdomain on Ngrok.com and it’ll be accessible over both HTTP and HTTPS (Secure).

 

 

Running Multiple Tunnels Simultaneously:

We are using an extensive feature of Ngrok i.e., we can run multiple tunnels simultaneously and a few tunnel configuration is shown in the following YML configuration file.

 

 Sample XML code
Sample XML code for running multiple tunnels simultaneously (.yml file)

 

We can start all four tunnels simultaneously by using Ngrok start command followed by the names of the tunnels we want to start:

 

Ngrok start command

 

The Output Terminal will look something like this:

 

The Output Terminal


Request inspection with the Web Interface:

Web Interface is accessible on http://127.0.0.1:4040 using which we can inspect all the http traffic requests over the tunnel. We can also replay them to make the debugging quicker and easier.

 

Sample Web Interface for HTTP Traffic Inspection: 

Sample Web Interface for HTTP Traffic Inspection

 

Architecture after Ngrok deployment

Architecture after Ngrok deployment

 

 

Conclusion

All in all, this is an amazing, secure and powerful tool that helps to meet our business needs on right time.