How to modify HTTP request header in Selenium WebDriver with Java?

Question

I want to add "iv-user" request header to log into web app.
I'm using the newest ChromeDriver. Using extensions like ModHeader or ModifyHeaders is not working in this case. Fiddler is working, but I need a solution that will allow me to change accounts during the test case execution.

Can someone help me with this, please?

Now, I'm trying to use BrowserMob Proxy in Embedded mode, to achieve it, but something is not working. Proxy is running and ChromeDriver too, request header is set and sent to the app server, but after checking network log in browser, this custom header is not received by it.

Class Case:

@BeforeEach
public void setUp() throws Exception {
 LOGER.setLevel(Level.WARNING);
 loadProperties();
 if (driver != null) {
 driver.quit();
 }
 BrowserMobProxy proxy = new BrowserMobProxyServer();
 proxy.start(0);
 Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);
 proxy.addRequestFilter((request, contents, messageInfo)->{
 request.headers().add("iv-user", "login");
 System.out.println(request.headers().entries().toString());
 return null;
 });
 chromeOptions = new ChromeOptions();
 String proxyOption = "--proxy-server=" + seleniumProxy.getHttpProxy();
 chromeOptions.addArguments(proxyOption);
 chromeOptions.addArguments("-incognito");
 chromeOptions.addArguments("disable-infobars");
 chromeOptions.setCapability(CapabilityType.PLATFORM_NAME, Platform.WINDOWS);
 chromeOptions.setCapability(CapabilityType.UNEXPECTED_ALERT_BEHAVIOUR, UnexpectedAlertBehaviour.ACCEPT);
 chromeOptions.setCapability(CapabilityType.ACCEPT_SSL_CERTS, true);
 System.setProperty("webdriver.chrome.driver", CHROMEDRIVER_PATH);
 driver = new ChromeDriver(chromeOptions);
 actions = new Actions(driver);
 driver.manage().timeouts().implicitlyWait(IMPLICITLY_WAIT_TIME_OUT, TimeUnit.SECONDS);
 driver.manage().window().maximize();
 new WebDriverWait(driver, 30);
 wait = new FluentWait<WebDriver>(driver)
 .withTimeout(ConstConfig.FLUENTWAIT_TIMEOUT_SEC, TimeUnit.SECONDS)
 .pollingEvery(1, TimeUnit.SECONDS)
 .ignoring(StaleElementReferenceException.class)
 .ignoring(NoSuchElementException.class)
 .ignoring(ElementNotVisibleException.class);
 jsExecutor = (JavascriptExecutor) driver;
}

Class Test:

public class Test_Proxy extends Case {
@Test
public void test_Proxy() throws Exception {
 try{
 driver.get(TESTAPP_URL);
 MainMenu mainMenu = new MainMenu();
 //..........
 } finally {//....................}}}

• Sergey Kochergan: sqa.stackexchange.com/questions/29473/…

Answer 1

Here is the completed example that demonstrates how you can modify requests from your Selenium tests. To demonstrate one I have created a sample REST mock service via SoapUI that would return just a {"SUCCESS"} message for any response.

The test code (TestNg is used for test running, and borwsermob-proxy for proxying requests):

public class MiscTests {
 WebDriver driver;
 @BeforeTest
 public void setUp(){
 BrowserMobProxy proxy = new BrowserMobProxyServer();
 proxy.start(0);
 Proxy seleniumProxy = ClientUtil.createSeleniumProxy(proxy);
 // put our custom header to each request
 proxy.addRequestFilter((request, contents, messageInfo)->{
 request.headers().add("my-test-header", "my-test-value");
 System.out.println(request.headers().entries().toString());
 return null;
 });
 // Setting up Proxy for chrome
 ChromeOptions opts = new ChromeOptions();
 String proxyOption = "--proxy-server=" + seleniumProxy.getHttpProxy();
 opts.addArguments(proxyOption);
 System.setProperty("webdriver.chrome.driver", "E:/Dev/WebDrivers/chromedriver.exe");
 driver = new ChromeDriver(opts);
 }
 @Test
 public void testProxifying(){
 driver.get("http://localhost:8080/");
 Assert.assertEquals(driver.findElement(By.xpath("//body")).getText(), "{\"SUCCESS\"}");
 }
 @AfterTest
 public void tearDown(){
 if(driver != null){
 driver.quit();
 System.out.println("Driver was instantiated. Quitting..");
 }else{
 System.out.println("Driver was null so nothing to do");
 }
 }
}

In above code you set up a proxy and configure the request filter that process all the outgoing requests. In that filter you add a sample header (modify according to your purpose). In SoapUI log we can see that our added header has successfully came to the server.

enter image description here

P.S.: - Here you can find more examples of browsermob-proxy and selenium

• M. Murawski: Alexey, I've used your code, but it is not working. BMP starts, and Chrome is running, but network log in browser shows, that this header is not sent in the request. I'll update the question.
• Alexey R.: Your log in browser won't show the new header because it is altered after the request leaves the browser.
• M. Murawski: Okay, but when I'm using manually ModHeader extension, it shows up in log, and logs me in. (Doesn't work with ChromeDriver).
• Alexey R.: @M.Murawski You can check with httpbin.org/get. Test with System.out.println(driver.getPageSource()); It responds with the headers which come from your client. I tested https connection in that way and it worked in my case. If in your case your header will be returned, then your issue has nothing to do with headers.
• Harsh: Just to shout it out... This doesn't work on Safari..

Answer 2

Selenium doesn't have API to do that. You need to use something external.

As Alexey suggested one of the solutions would be setting up proxy like this.

• M. Murawski: Michał, I'm already using BrowserMobProxy as you recommended, but now I have to develop a method to modify one of the request headers. Maybe you could help me with this?

Answer 3

You cannot achieve it by Selenium, but as you mentioned you are using BrowserMobProxy , I would recommend you to try once below .

1. Apache Module mod_headers ( Refer : http://httpd.apache.org/docs/current/mod/mod_headers.html#page-header )

2. You can also install Fiddler (http://www.fiddler2.com/fiddler2/) which is very easy to install (easier than Apache for example). After launching it, it will register itself as system proxy. Then open the "Rules" menu, and choose "Customize Rules..." to open a JavaScript file which allow you to customize requests. To add a custom header, just add a line in the OnBeforeRequest function:

   oSession.oRequest.headers.Add("MyHeader", "MyValue");

Hope this helps.

• M. Murawski: I've tried Fiddler, and it works, but I need something, which will be working in Continuous Integration (Bamboo), and will allow relogging to another account during test case execution.
• Prasad_Joshi: Its really lengthy to explain it here with very little background hard to help.

Answer 4

With the solutions already discussed above the most reliable one is using Browsermob-Proxy

But while working with the remote grid machine, Browsermob-proxy isn't really helpful.

This is how I fixed the problem in my case. Hopefully, might be helpful for anyone with a similar setup.

1. Add the ModHeader extension to the chrome browser

How to download the Modheader? Link

ChromeOptions options = new ChromeOptions();
options.addExtensions(new File(C://Downloads//modheader//modheader.crx));
// Set the Desired capabilities 
DesiredCapabilities capabilities = new DesiredCapabilities();
capabilities.setCapability(ChromeOptions.CAPABILITY, options);
// Instantiate the chrome driver with capabilities
WebDriver driver = new RemoteWebDriver(new URL(YOUR_HUB_URL), options);

2. Go to the browser extensions and capture the Local Storage context ID of the ModHeader

Capture ID from ModHeader

3. Navigate to the URL of the ModHeader to set the Local Storage Context

.

// set the context on the extension so the localStorage can be accessed
driver.get("chrome-extension://idgpnmonknjnojddfkpgkljpfnnfcklj/_generated_background_page.html");
Where `idgpnmonknjnojddfkpgkljpfnnfcklj` is the value captured from the Step# 2

4. Now add the headers to the request using Javascript

.

 ((Javascript)driver).executeScript(
 "localStorage.setItem('profiles', JSON.stringify([{ title: 'Selenium', hideComment: true, appendMode: '', 
 headers: [ 
 {enabled: true, name: 'token-1', value: 'value-1', comment: ''},
 {enabled: true, name: 'token-2', value: 'value-2', comment: ''}
 ], 
 respHeaders: [],
 filters: []
 }]));");

Where token-1, value-1, token-2, value-2 are the request headers and values that are to be added.

5. Now navigate to the required web-application.

   driver.get("your-desired-website");

Answer 5

I was not able to see the added parameterin network tab. later developer told that I need to add to response and not to request. now I can see in chrome network tab

BrowserMobProxy proxy = new BrowserMobProxyServer();
//proxy.setTrustAllServers(true);
proxy.start();
Map<String, String> headers = new HashMap<String, String>();
headers.put("x-stack-b", "true");
proxy.addResponseFilter((response, contents, messageInfo)->{
response.headers().add("x-stack-b", "true");
System.out.println("filter: " +response.headers().entries().toString()); return ;
});

Answer 6

How can set up the same in VBA script? can anyone help on this? I need pass few headers so that akamai won't treat me as bot once i run the application

• Lee Jensen: This does not really answer the question. If you have a different question, you can ask it by clicking Ask Question. To get notified when this question gets new answers, you can follow this question. Once you have enough reputation, you can also add a bounty to draw more attention to this question. - From Review