How To Run Selenium Web Driver Behind A Proxy Server Which Needs Authentication In Python

At present this is my code, but webDriver is showing a pop-up to enter proxy credentials and I don't want this annoying situation, This is not the first time the same question appe

Solution 1:

Selenium can't handle with basic authentication neither it works well with popups. But this problem is resolvable. As a solution that worked to me (I found it here) is to add a browser extension that does the authentication for Selenium. It's quite simple. Here is how it works for Chrome and Python:

1. Create a zip file proxy.zip containing two files:

background.js

var config = {
    mode: "fixed_servers",
    rules: {
      singleProxy: {
        scheme: "http",
        host: "YOU_PROXY_ADDRESS",
        port: parseInt(YOUR_PROXY_PORT)
      },
      bypassList: ["foobar.com"]
    }
  };

chrome.proxy.settings.set({value: config, scope: "regular"}, function() {});

functioncallbackFn(details) {
    return {
        authCredentials: {
            username: "YOUR_PROXY_USERNAME",
            password: "YOUR_PROXY_PASSWORD"
        }
    };
}

chrome.webRequest.onAuthRequired.addListener(
        callbackFn,
        {urls: ["<all_urls>"]},
        ['blocking']
);

Don't forget to replace YOUR_PROXY_* to your settings.

manifest.json

{"version":"1.0.0","manifest_version":2,"name":"Chrome Proxy","permissions":["proxy","tabs","unlimitedStorage","storage","<all_urls>","webRequest","webRequestBlocking"],"background":{"scripts":["background.js"]},"minimum_chrome_version":"22.0.0"}

2. Add the created proxy.zip as an extension

Python Code:

from selenium import webdriver
from selenium.webdriver.chrome.optionsimportOptions

chrome_options = Options()
chrome_options.add_extension("proxy.zip")

driver = webdriver.Chrome(executable_path='chromedriver.exe', chrome_options=chrome_options)
driver.get("http://google.com")
driver.close()

That's it. For me that worked like a charm. If you need to create proxy.zip dynamically or need PHP example then go to the original post

Solution 2:

I know its pretty late replying for your question but recently I started working with Python, and was trying to do the same and did something like this to handle this situation.

To run selenium web driver behind proxy server

1. Need to create a firefox profile, where "autoauth" add-on should be installed.
2. Try to save proxy server username & password by hitting a URL manually.
3. Firefox profile will save the credential of proxy server with help of autoauth
4. In script call that particular Firefox profile.
5. Set all the preferences to define proxy server details.
6. Assign Firefox profile in to browser's instance
7. Hit any URL, Below is the running example

P.S. : Remove all proxy setting from internet options, script will use it automatically

So technically here you will not send proxy username & password, you'll save those credential in firefox and invoke that particular firefox profile.

Hope you have already solved your problem long back, but in case its still there, this might help you. :)

Solution 3:

Refer to this path on how to access the firefox profile: http://software-testing-tutorials-automation.blogspot.jp/2014/05/how-to-create-and-use-custom-firefox.html

Basically you need to create a profile and use that profile in firefox. Now the behaviour of your profile will also be the same when you run in via selenium. So if you're profile will prompt proxy auth, then it will also display when you open it via selenium. So you have to find a way by using add-ons to hide the authentication using that profile. After you have successfully done that, it will be the same now with Selenium.