Project

General

Profile

Actions

Bug #1053

closed

Cloudflare blocks access for some Privacy Browser users

Added by jindam vani 10 months ago. Updated 3 months ago.

Status:
Closed
Priority:
3.x
Start date:
08/07/2023
Due date:
% Done:

0%

Estimated time:

Description

  • samsung internet_ 21.0, jquarks viewer_ 1.10.19: successful login
  • 3.15.1 fdroid_ infinite loop verfication, unable to login_ gitlab.com
  • os: android 9

Files

GitLab Domain Settings.png (213 KB) GitLab Domain Settings.png Soren Stoutner, 08/07/2023 07:55 AM
error.png (85 KB) error.png gitlab error jindam vani, 08/07/2023 04:41 PM
privacy.png (155 KB) privacy.png default webview & other settings jindam vani, 08/07/2023 04:41 PM
domain.png (155 KB) domain.png gitlab settings_ domain jindam vani, 08/08/2023 04:05 AM
git.png (84.8 KB) git.png gitlab verification jindam vani, 08/08/2023 04:06 AM
Applied Domain Settings.png (158 KB) Applied Domain Settings.png Soren Stoutner, 08/08/2023 07:41 AM
applied.png (96.4 KB) applied.png url green jindam vani, 08/08/2023 08:27 AM
Privacy Browser Android 3.15.1 Version.png (354 KB) Privacy Browser Android 3.15.1 Version.png Soren Stoutner, 08/11/2023 10:14 AM
Privacy Browser Android 3.15.1 Version.png (293 KB) Privacy Browser Android 3.15.1 Version.png about screen screenahot jindam vani, 08/11/2023 11:09 AM
Privacy Browser Android 3.15.1 Version.png (288 KB) Privacy Browser Android 3.15.1 Version.png fdroid app about jindam vani, 08/11/2023 05:42 PM
test_Privacy Browser Android 3.15.1 Version.png (292 KB) test_Privacy Browser Android 3.15.1 Version.png test app about jindam vani, 08/11/2023 05:43 PM
jQuarks Permissions.png (120 KB) jQuarks Permissions.png Soren Stoutner, 08/21/2023 10:24 AM
Actions #1

Updated by Soren Stoutner 10 months ago

Logging into GitLab works for me with the domain settings in the attached screenshot.

Updated by jindam vani 10 months ago

  • unfortunately its still not working
  • perhaps i need to use user agent similar to your attached file?
Actions #3

Updated by Soren Stoutner 10 months ago

You need to make the domain name *.gitlab.com, which applies the domain setting to all gitlab subdomains. Your screenshot shows *gitlab.com (missing the first period).

Updated by jindam vani 10 months ago

  • for some reason its struck at verification process
Actions #5

Updated by Soren Stoutner 10 months ago

From your screenshots, you can see that the domain settings are not being applied, because the background of your URL is white. Check to see if there are any whitespaces after your domain name in the domain settings. Once you get things right, the domain settings will be applied and the URL background will turn green as shown in this screenshot.

Actions #6

Updated by jindam vani 10 months ago

  • now my url in green
  • still no luck ;)
Actions #7

Updated by Soren Stoutner 10 months ago

Note that it takes a few seconds to get past the screen in your screenshot above. If waiting doesn't work, please delete all your domain settings that relate to GitLab. Then record a video similar to the one linked below. Watching all the steps will help me figure out where it is going wrong for you.

https://nextcloud.stoutner.com/s/p6f9JBYSiHfjELF

Actions #8

Updated by jindam vani 9 months ago

Actions #9

Updated by Soren Stoutner 9 months ago

Perhaps Cloudflare doesn't like some aspect of the WebView default user agent on your device. Try using domain settings to set a custom user agent to be the following:

Mozilla/5.0 (Linux; Android 13; Pixel 5 Build/TQ3A.230705.001; wv) AppleWebKit/537.36 (KHTML, like Gecko) Version/4.0 Chrome/116.0.0.0 Mobile Safari/537.36
Actions #11

Updated by Soren Stoutner 9 months ago

I wondered if there was something about Android 9 that was making Cloudflare unhappy. So, I loaded up Android 9 in an emulator and tried it. With the default WebView that shipped with Android 9 (69.0.3497.100), I did have to check the "Verify you are human" box (which I don't have to do on my Pixel 5), but then it let me in. Updating WebView to 115.0.5790.166 let me in without the need to check the box.

At this point I wonder if you perhaps changed one of Privacy Browser's system-wide settings in a way that makes Cloudflare unhappy. I can't imagine any of them doing that, but I suppose it is possible. You can check if this is the case by installing the Alt (testing) version of Privacy Browser from the following URL:

https://download.stoutner.com/privacy-browser/Privacy Browser Alt 3.15.1.apk

This will install alongside your current version and has a different icon with the word TEST.

Open it up and don't make any other changes besides the Domain Settings we have done previously. If you have the same results as before, send me a screenshot of About > Version from inside the app, which might provide some clues. Attached is a screenshot from the Android 9 emulator that was able to get to the login screen.

Actions #12

Updated by jindam vani 9 months ago

  • done nothing, except creating domain settings
Actions #13

Updated by jindam vani 9 months ago

  • i have no issues with samsung internet, jquarks viewer; when i created this issue & now also
  • to be on safe side, upgraded chrome to latest version
  • i am wondering why two browsers depending on same_ webview, behaving vastly differently
Actions #14

Updated by Soren Stoutner 9 months ago

It is very odd, because everything I see indicates that your setup is functionally similar to mine, but Cloudflare lets me log on to GitLab and blocks you.

As a curiosity, try switching the WebView provider from Chrome to Android System WebView (after you do this you will want to install the updates for Android System WebView). There is a video showing how to do this at https://www.stoutner.com/webview/.

Actions #16

Updated by Soren Stoutner 9 months ago

It seems there is something about your setup that makes Cloudflare more suspicious of you than of any of the devices I have tested.

The only two possibilities I can think of are that either 1) Samsung has made some modification to Android on that device that makes WebView behave differently than it generally does. Or, 2) there is something about your IP address that makes Cloudlfare extra suspicious of your device.

There are a couple of things that make 1 unlikely to be true. The most significant is that jQuarks works, which also uses WebView. Additionally, as a test to see if Samsung devices in general have problems, I used a Samsung Galaxy Tab A7, and it didn't have any any with loading GitLab in Privacy Browser.

The principle difference I can see between Privacy Browser and jQuarks is that they use different ad blocking lists. These don't generally cause problems with GitLab, but it might be worth checking to see if disabling EasyList, EasyPrivacy, Fanboy's annoyance list, Fanboy's social blocking list, UltraList, and UltraPrivacy in Domain Settings for *.gitlab.com makes any difference.

The second possibility is that Cloudflare is particularly suspicious of your IP address. Do you have the ability to try a different device from your IP address or your current device from a different IP address?

Actions #17

Updated by jindam vani 9 months ago

  • unable to complete verification after disabling all lists
  • failure apps from fdroid: * foss browser v 9.11 * einkbro v 10.11.0 * monocles browser v 1.7.1 * smartcookieweb v 16.3 * fulguris web  browser v 1.9.8
  • success apps from fdroid * jquarks viewer v 1.0-19 * mull v 116.2.0 [ etp: strict, standard ] * fennec fdroid v 116.2.0 [ etp: strict, standard ]
Actions #18

Updated by v ... 9 months ago

For the record, I also tried to display the GitLab's login page, and was also stuck in the same infinite loop.

Tested on Privacy Browser Android v3.15.1 "standard" (prod) and "alt" (test), recently downloaded, with my VPN client enabled and then disabled.

Enabled options :
  • JavaScript
  • Cookies
  • DOM Storage
  • User Agent set as suggested in #note-9
Disabled options :
  • all filter lists
Actions #19

Updated by Soren Stoutner 9 months ago

Out of curiosity, I tried logging in with jQuarks. For me, it didn't work (see the linked video below). I was using the app's default settings. When you were able to log in using jQuarks, did you have to make any changes to the settings first?

https://nextcloud.stoutner.com/s/B3teiSHkNNdXWJk

Actions #20

Updated by jindam vani 9 months ago

When you were able
to log in using jQuarks, did
you have to make any changes
to the settings first?

yes, disabled location access.
for any app immediately after
installation i disable all
other permissions except
storage.

Actions #21

Updated by Soren Stoutner 9 months ago

Before doing my testing, I had not allowed any special permissions for jQuarks (it didn't ask for any), meaning that location was disabled (see screenshot).

Assuming that you didn't change any of the app settings inside of jQuarks, this means that jQuarks can access GitLab for you, but for me with the same settings it doesn't work. Conversely, Privacy Browser can access GitLab for me, but for your with the same settings it doesn't work.

That being so, it would lead me to believe that Cloudflare is using a different set of rules, tests, or criteria to decide if our devices will be allowed through. My guess is that difference is determined by our IP addresses, but perhaps there is something else about our devices that Cloudflare is using to differentiate us.

Cloudflare is fairly opaque about what criteria they use to determine if they should allow a connection, and that criteria changes over time, so even if you knew what it was yesterday you might not know today. In any case, there doesn't seem to be anything I can do as the developer of Privacy Browser that will make a difference, especially because Cloudflare really likes to spy on you, so as the 4.x series comes out and Privacy Browser gets better and better at stopping all that type of spying (just you wait until I completely remove the user agent by default) you will increasingly need to decide between turning off all of Privacy Browser's protections to access Cloudflare, using an insecure browser that doesn't have these protections to access Cloudflare, or not accessing Cloudflare at all.

Actions #22

Updated by Soren Stoutner 9 months ago

  • Subject changed from infinite loop verfication_ gitlab.com to Cloudflare blocks access for some Privacy Browser users
Actions #23

Updated by ask low 5 months ago

@v ... @jindam vani The issue was strangely Images. Browserwide I've disabled it & enabling it for gitlab worked. Other configs like JS/DOM/Cookies/Filters have no effect on this.

Actions #24

Updated by Soren Stoutner 5 months ago

The issue is probably different for different people over time as Cloudflare is constantly changing their algorithms. Sometimes there might be nothing a user can do to get through, but other times there are unexpected things that work, like enabling the loading of images (something bots often don't do, so a check Cloudflare is interested in).

If anyone finds something that is helpful, putting it here (or in the forum) is beneficial so that other users have a list of things they can try.

Actions #25

Updated by ask low 3 months ago

I'm sorry but if this issue is not in FOSS Browser or other webview wrappers, I donno how this is not specific to PB.
https://linuxiac.com/
Works on FOSS Browser, but stuck in cloudfare human verification on PB. Ticking the box doesn't even let me in either. It's just recursive.

Actions #26

Updated by Soren Stoutner 3 months ago

FOSS Browser behaves differently than Privacy Browser Android, which Cloudflare can detect. If Cloudflare chooses to let FOSS Browser through there is nothing I can do about it because Privacy Browser doesn't mimic those behaviors on purpose.

For example, FOSS Browser sends the following header, even if you have disabled the X-Requested-With header using WebView DevTools.

X-Requested-With: com.duckduckgo.mobile.android

In the end, Cloudflare wants to spy on you. They don't like browsers that won't let them spy on you. As Privacy Browser gets better and better they will move from sometimes blocking it to always blocking it. At that point, users have two options. 1) Use a different browser that lets Cloudflare spy on them to access websites hosted on Cloudflare. Or, 2) don't visit sites hosted on Cloudflare.

Actions #27

Updated by ask low 3 months ago

Got it. I believe option 2 but I mean, literally more & more articles started hosting their sites using cloudfare DNS servers. It's just getting exhaustive each time I'm encountering it on PB.

Actions

Also available in: Atom PDF