Privacy Browser Android

This is because Walmart doesn't like the null `X-Requested-With` header.

https://www.stoutner.com/the-x-requested-with-header/

I would consider this to be a bug in the Walmart website. You can contact their webmaster if you think you can get any traction out of them.

Ok. Well I dont know how to contact them without access to their website lol. but anyway aside from that I have a bug report for you.

v3.8.1

Import failed: android.database.sqlite.SQLiteException: no such table bookmarks (Code 1 SQLITE_ERROR):,while compiling:SELECT *FROM bookmarks

Also I noticed you removed the option for 3rd party cookies in 3.8.1 does this mean 3rd party cookies will be enabled on every site that requires cookies enabled?

Please create a bug report regarding your settings import problem, including information about how your settings file was created, and, if there is no sensitive information in it, an attached copy of the file that is producing the import error.

The answer to your question about third-party cookies can be found at https://redmine.stoutner.com/issues/601. There is also some information at https://www.stoutner.com/privacy-browser-3-8/.

Ok done.

Walmart responded by asking me to try a different web browser so I downloaded amother webview browser here https://f-droid.org/en/packages/de.baumann.browser/

This browser has no problem with the walmart.com site.

That doesn't surprise me. As explained in the X-Requested-With link posted above, Privacy Browser sends the `X-Requested-With` header with a null value (eventually, in the 4.x series, the `X-Requested-With` header will be removed entirely). There is nothing wrong with sending a header with a null value, and most web servers just ignore it. But it isn't very common, and some websites block traffic from browsers that send headers with null values, often unintentionally because their header parsers are poorly designed. As I said earlier, I consider that a bug in the web server and not in Privacy Browser.

You can see all the headers that a browser sends and see if any of them are null by visiting https://browserleaks.com/ip.

Hmmm so what is expected in that header? The user agent?

-- that other browser sends x-requested-with: com.duckduckgo.mobile.android

Why doesn't Privacy Browser just send whatever user agent is the one in the settings?

1. That header shouldn't be sent at all for normal traffic (it has a specific purpose relating to AJAX requests), and most browsers do not send it by default.
2. Google abuses the header for apps that use Android's WebView and, by default, sends the ID of the app. For Privacy Browser that is com.stoutner.privacybrowser.standard.
3. It is possible to programatically override the value of the header, but not to remove it. I have chosen to change it to a null value as that provides the least fingerprinting information. FOSS Browser used to also change it to a null value, but has now decided to change it to com.duckduckgo.mobile.android.
4. As I mentioned earlier, in the 4.x series with the introduction of Privacy WebView, I intend to completely remove the header.
5. Any website that has issues with a null header value needs to fix their design. Just wait until I do things that will really stress them out, like completely dropping the user agent by default in the 4.x series (I would already do so, but Android's WebView currently doesn't allow you to not send a user agent, neither does it allow you to send a null value for the user agent).

Hi Stoutner,
With this issue, what do you recommend the app user do from a process perspective?
1, Raise an issue with the website to fix the `X-Requested-With` header with a null value
2, Wait for the website to get fixed (or not with stubborn websites admins)
3, While waiting, use a different, less privacy focused, browser app to use the website
4, If the website doesn't get fixed then the user will always need to use a different, less privacy focused, browser app

From a users point of view, this will become quite messy. For example, today I was trying to search for a PS4 game on currys.co.uk and couldn't use Privacy Browser due to the same error as this. I had to open Chrome on my phone to do the search (which was logged in as my google account) and now I'm seeing ads for PS4 games on my laptop. It was a shame I couldn't use Privacy Browser which would have stopped this tracking and ad targeting.

Would an option be to add a setting in Privacy Browser that could be "switched on" (like cookies) for not sending `X-Requested-With` header with a null value?

It would be possible to add a setting to modify the `X-Requested-With` header, but I am not inclined to do so. The primary reason is that I consider this to be a bug in the website, so it should be fixed there. But, the bigger issue is that Privacy Browser, because of its core design philosophy, will, over time, cause lots of problems with lots of websites that don't want to respect user privacy. One of the core things I want to accomplish with Privacy Browser is to change what is considered acceptable in web design as it relates to user privacy, and you can't do that unless you break bad websites.

Some things, like blocking all third-party requests, currently breaks so many websites that it isn't enabled by default. But it will be some day.

Other things, like allowing mixed content (HTTP links inside an HTTPS website) is so fundamentally wrong that there is no way for a user to override the default settings. Websites that do this are 1) in a small minority, and 2) could be easily redesigned to fix the problem. Websites that have problems with headers that contain null values fit into this category.

From an end user perspective, I expect that most people who use Privacy Browser will also have another browser installed. That way, they can use the other browser when they want to visit a website that does not respect their privacy. The need to switch browsers to use a particular website becomes a constant reminder to the user that such website is not secure, and might eventually prompt them to use a better designed competitor.

My long term goal is for Privacy Browser to account for 20% of worldwide browser usage. At that level, most web designers will modify their webpages so they work with Privacy Browser's default settings.

Thanks for coming back to me again.
I understand and respect what you're trying to do, ultimately, you're trying to make the web better for us end users.

Could you give me a sentence or two on what I should be asking the currys.co.uk website to fix to overcome "The requested URL was rejected." issue please?
I'll try and find a support link and make a request to them.

I've found pcworld.co.uk to have the same issue but I think they're part of the same group so could be the same web admins.

I would probably point them to https://github.com/opnsense/core/issues/4917, include the error message you receive when visiting their site, and tell them they have a similar problem.

In the case of the OPNsense issue above, I initially thought it was a different problem, but it ended up being that their web server (Lighttpd) has a header parsing bug that causes problems with null values. In the case of Lighttpd it has been fixed in a future release: https://github.com/lighttpd/lighttpd1.4/commit/262561f.

I just thought I'd let you know that I had a response from currys.co.uk/pcworld.co.uk:

Dear customer,
Thank you for your email regarding your product. It has been assigned the reference number 3485989184; please quote this number in any future correspondence.
I have noted the fault in the call log.
I don?t know if and when a fix will be made available.
In the meantime I have tested the website using the android chrome browser and it works fine.
Should you have any further queries please don't hesitate to contact us either via email or by telephone on 0344 561 1234 (UK) / 1890 818 575 (ROI). Lines are open 24 hours a day, 7 days a week.
Kind regards,
Ali
Team Knowhow