Project

General

Profile

Actions

Feature #37

open

Competely remove the X-Requested-With header

Added by Soren Stoutner over 7 years ago. Updated about 2 years ago.

Status:
New
Priority:
4.x
Start date:
06/28/2016
Due date:
% Done:

0%

Estimated time:

Description

By default, WebView sends an X-Requested-With header with the application ID (com.stoutner.privacybrowser.standard) with every page load. It looks like it is possible to change the header, but not to leave it out (Lightning sends the header with a null value).

http://stackoverflow.com/questions/17832313/android-disable-x-requested-with-header-in-webview

http://stackoverflow.com/questions/7610790/add-custom-headers-to-webview-resource-requests-android#comment10183596_8209272

At a minimum, it would be nice to allow the user to set a custom header. Ultimately, we might need to embed a custom WebView to get full control of the functionality.

Actions #1

Updated by Soren Stoutner over 7 years ago

  • Description updated (diff)
Actions #2

Updated by Soren Stoutner over 7 years ago

  • Description updated (diff)
Actions #3

Updated by Soren Stoutner over 7 years ago

  • Status changed from New to In Progress

This is partially fixed. The initial request to each website loads `X-Requested-With` paired with a null value.

However, requests for resources like images and css load `X-Requested-With:com.stoutner.privacybrowser.standard`.

This is because Google wants to personally make my life difficult.

http://stackoverflow.com/questions/38412965/adding-custom-header-to-webview-by-adding-headers-to-webresourcerequests-header

Actions #4

Updated by Soren Stoutner over 7 years ago

  • Subject changed from Customize X-Requested-With Header to Customize `X-Requested-With` Header
Actions #6

Updated by Soren Stoutner over 7 years ago

Beginning with commit https://git.stoutner.com/?p=PrivacyBrowser.git;a=commitdiff;h=87cd85777034a3b7627b68b3d60004fb20198727 Privacy Browser will send a null value for `X-Requested-With` on initial web requests. This will be included in Privacy Browser 1.9.

https://www.stoutner.com/the-x-requested-with-header/

Actions #7

Updated by Soren Stoutner almost 7 years ago

  • Priority changed from 2 to 3.x
  • Parent task set to #86

This cannot be fully fixed until Privacy WebView replaces Android's WebView.

Actions #8

Updated by Soren Stoutner over 6 years ago

  • Priority changed from 3.x to 4.x
Actions #9

Updated by Soren Stoutner over 6 years ago

  • Parent task deleted (#86)
Actions #10

Updated by Florian Heimgaertner over 4 years ago

Soren Stoutner wrote:

Beginning with commit https://git.stoutner.com/?p=PrivacyBrowser.git;a=commitdiff;h=87cd85777034a3b7627b68b3d60004fb20198727 Privacy Browser will send a null value for `X-Requested-With` on initial web requests. This will be included in Privacy Browser 1.9.

https://www.stoutner.com/the-x-requested-with-header/

Using a null value is probably not the best approach. Keys without values are considered malformed headers by some web servers leading to HTTP 400 errors.

My suggestion is to disable the header override until X-Requested-With can be completely removed (or make it optional, like lightning does). Currently the privacy benefits are near zero as the header override only applies to initial requests initiated via bookmarks or manually typed URLs (WebView 76.0.3809.89). Pages opened via link receive the complete header (like requests for resources as mentioned above).

Actions #11

Updated by Soren Stoutner over 4 years ago

Can you provide an example of a webserver that has difficulty with a null header value? In all my testing, I have not seen one.

Regardless, I would consider that to be a bug in the webserver that should be fixed there. I am not inclined to change Privacy Browser's current behavior to accommodate it.

Actions #12

Updated by Frank Doepper almost 4 years ago

https://www.berlin.de/ sends "418 I'm a teapot" when treated with an empty 'x-requested-with:' header

Actions #13

Updated by Frank Doepper almost 4 years ago

... and https://www.voebb.de/ sends "400 Bad Request". There seem to be more webservers in the wild than last year refusing to handle an empty x-requested-with

Actions #14

Updated by Soren Stoutner almost 4 years ago

  • Subject changed from Customize `X-Requested-With` Header to Competely remove the X-Requested-With header
  • Status changed from In Progress to New

Both of those websites do appear to be blocking Privacy Browser because they don't like the X-Requested-With header. However, as I said earlier, I would consider this a bug with the website and not with Privacy Browser.

Actions #15

Updated by Soren Stoutner almost 3 years ago

Apparently Lighttpd recently decided to block access to browsers sending empty headers. Either this change or something else they did affected Safari on iOS as well.

It looks like they are going to revert the decision, which is good. Blocking empty headers provides no security or other benefit to a web server.

https://github.com/opnsense/core/issues/4917

Actions #16

Updated by Soren Stoutner about 2 years ago

Lighttpd has fixed the issue, so it is no longer a problem for websites they are hosting. However, I have received a report of other websites that have issues. Even though I still consider this a bug in the website itself, I think I am going to create a setting where users can revert the X-Requested-With behavior if desired.

https://redmine.stoutner.com/issues/819

Actions

Also available in: Atom PDF