serrebi/BlindRSS

Watch

Fork 0

mirror of https://github.com/serrebidev/BlindRSS.git synced 2026-08-13 09:49:26 -07:00

Feed parser fails to extract articles from valid feeds in multiple RSS/Atom formats #29

New issue

Closed

opened 2026-06-28 14:25:35 -07:00 by serrebi · 7 comments

serrebi commented

2026-06-28 14:25:35 -07:00

Owner

The parser must be extended to reliably handle a wider range of feed formats.

Formats that should be supported to ensure wide compatibility:

RSS 0.90, 0.91, 0.92-0.94
RSS 1.0 (RDF-based)
RSS 2.0
Atom 1.0 (RFC 4287)

Examples of problematic feeds provided by major online services:

Steps to reproduce

Add any of the listed feeds to BlindRSS.
Wait for the feed to load.
Observe that no articles are extracted (the UI may show “No articles found”).
Confirm the feed works in another aggregator (e.g., Feedly, Inoreader).

The current parser in BlindRSS is unable to extract articles from several valid feeds. The feeds are confirmed to work in other aggregators, but BlindRSS shows “No articles found” because it fails to parse the content. The parser must be extended to reliably handle a wider range of feed formats. Formats that should be supported to ensure wide compatibility: * RSS 0.90, 0.91, 0.92-0.94 * RSS 1.0 (RDF-based) * RSS 2.0 * Atom 1.0 (RFC 4287) Examples of problematic feeds provided by major online services: * <https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/> * <https://gitlab.gnome.org/GNOME/orca/-/tags?format=atom> * <https://getgrav.org/blog.rss> ### Steps to reproduce 1. Add any of the listed feeds to BlindRSS. 2. Wait for the feed to load. 3. Observe that no articles are extracted (the UI may show “No articles found”). 4. Confirm the feed works in another aggregator (e.g., Feedly, Inoreader).

serrebi commented

2026-06-28 14:56:21 -07:00

Author

Owner

Thanks for the detailed report. This is fixed and released in v1.70.2:

https://github.com/serrebidev/BlindRSS/releases/tag/v1.70.2

What changed:

Local feed refresh now normalizes entry content, links, dates, and identity across RSS 0.90, 0.91, 0.92, 0.93, 0.94, RSS 1.0, RSS 2.0, and Atom 1.0.
Legacy entries without a guid or link are no longer discarded; BlindRSS generates a stable per-feed entry ID from the available title/date/content.
Feeds that return HTTP 406 to browser-style headers, including the GitLab Atom example, are retried with generic feed-reader headers.
Added regression coverage for the listed RSS/Atom formats and the 406 retry path.

I also verified the three reported feeds through the local provider before release:

APKMirror Android Accessibility Suite feed: 10 articles inserted
GNOME Orca tags Atom feed: 20 articles inserted
Grav blog RSS feed: 10 articles inserted

The Windows release assets are published now. The macOS/Linux release workflow has been dispatched and should attach those platform assets when it completes.

Thanks for the detailed report. This is fixed and released in v1.70.2: https://github.com/serrebidev/BlindRSS/releases/tag/v1.70.2 What changed: - Local feed refresh now normalizes entry content, links, dates, and identity across RSS 0.90, 0.91, 0.92, 0.93, 0.94, RSS 1.0, RSS 2.0, and Atom 1.0. - Legacy entries without a `guid` or `link` are no longer discarded; BlindRSS generates a stable per-feed entry ID from the available title/date/content. - Feeds that return HTTP 406 to browser-style headers, including the GitLab Atom example, are retried with generic feed-reader headers. - Added regression coverage for the listed RSS/Atom formats and the 406 retry path. I also verified the three reported feeds through the local provider before release: - APKMirror Android Accessibility Suite feed: 10 articles inserted - GNOME Orca tags Atom feed: 20 articles inserted - Grav blog RSS feed: 10 articles inserted The Windows release assets are published now. The macOS/Linux release workflow has been dispatched and should attach those platform assets when it completes.

serrebi commented

2026-06-28 16:35:51 -07:00

Author

Owner

Unfortunately, the problem was not resolved for two of the three URLs:

At the same time, they open and display successfully in the browser.

Quick checks to perform first

HTTP request headers.Many HTTP clients send a non‑human‑like User‑Agent by default. Sites with protection (Cloudflare, WAF) immediately drop the connection or return a placeholder page. Fix: add mandatory header User-Agent: Mozilla/5.0 (...) and, preferably, Accept: application/xml,text/xml,application/rss+xml;q=0.9,/;q=0.8.
Redirect handling. Feeds often return 301/302 to the actual URL (e.g., HTTP to HTTPS, or short to full URL). If the client does not follow redirects, it may get an empty response or an error. Fix: enable automatic redirect following (usually allow_redirects=True or equivalent) and cap the maximum number of redirects (e.g., 5).
Timeouts and connection resets. TCP‑level errors (Connection reset by peer, timeout) often cause clients to throw exceptions without exposing the status or response body. Fix: wrap the request in try/catch, explicitly catch network errors, and log not only the message but also the exception type and status code (if available).
Encoding and binary data. A feed may be delivered in UTF‑8, ISO‑8859‑1, or with BOM. If the parser expects one encoding and receives another, it might return an empty structure or a parsing error. Fix: use auto‑detection of encoding (from headers, BOM, meta) or explicitly decode with a fallback strategy.
Compression (Gzip/Deflate). The server may send the feed compressed, while the client tries to parse the compressed bytes as XML. Fix: allow or explicitly handle Content‑Encoding: gzip/deflate (most HTTP clients enable this by default, but it’s worth verifying).
XML/RSS validity. Even if a browser displays the feed, the XML might be malformed (unclosed tags, invalid entities, HTML‑style self‑closing tags). Fix: use a strict XML parser with error handling and optional “soft” recovery, or validate the feed before parsing.
IP/rate‑based blocking. Some sites limit request frequency from a single IP. Fix: add random delay between requests, limit concurrency, keep a simple request counter, and log HTTP codes 429/403.
SSL and certificates. SSL issues (self‑signed certificates, outdated ciphers, hostname mismatch) lead to connection failures. Fix: verify that SSL verification is not disabled in production; ensure test environments reflect real conditions.
Feed format. Besides RSS (2.0, 0.9x) and Atom, mixed or non‑standard variants exist. Fix: support multiple feed types, detect the type from the XML structure, and use appropriate parsers.
Dynamic feeds and JS rendering. Some “feeds” are HTML pages that become RSS in the browser via JavaScript, or they are not real RSS but HTML with a feed link in meta. Fix: check Content‑Type and document structure; do not attempt to parse HTML as RSS.

####Recommended improvements

Add logging: URL, method, headers (excluding secrets), response code, request duration, exception type.
Implement retry logic with exponential backoff for transient errors (5xx, timeouts, Connection reset).
Limit concurrent requests per domain and add basic rate‑limiting.
Unify error handling: separate error types for “network”, “HTTP”, “parsing”, “invalid format”.
Test against a set of “problematic” feeds (including those that open in a browser but fail in the aggregator).

Unfortunately, the problem was not resolved for two of the three URLs: * <https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/> * <https://getgrav.org/blog.rss> At the same time, they open and display successfully in the browser. #### Quick checks to perform first 1. HTTP request headers.Many HTTP clients send a non‑human‑like User‑Agent by default. Sites with protection (Cloudflare, WAF) immediately drop the connection or return a placeholder page. Fix: add mandatory header User-Agent: Mozilla/5.0 (...) and, preferably, Accept: application/xml,text/xml,application/rss+xml;q=0.9,*/*;q=0.8. 2. Redirect handling. Feeds often return 301/302 to the actual URL (e.g., HTTP to HTTPS, or short to full URL). If the client does not follow redirects, it may get an empty response or an error. Fix: enable automatic redirect following (usually allow_redirects=True or equivalent) and cap the maximum number of redirects (e.g., 5). 3. Timeouts and connection resets. TCP‑level errors (Connection reset by peer, timeout) often cause clients to throw exceptions without exposing the status or response body. Fix: wrap the request in try/catch, explicitly catch network errors, and log not only the message but also the exception type and status code (if available). 4. Encoding and binary data. A feed may be delivered in UTF‑8, ISO‑8859‑1, or with BOM. If the parser expects one encoding and receives another, it might return an empty structure or a parsing error. Fix: use auto‑detection of encoding (from headers, BOM, meta) or explicitly decode with a fallback strategy. 5. Compression (Gzip/Deflate). The server may send the feed compressed, while the client tries to parse the compressed bytes as XML. Fix: allow or explicitly handle Content‑Encoding: gzip/deflate (most HTTP clients enable this by default, but it’s worth verifying). 6. XML/RSS validity. Even if a browser displays the feed, the XML might be malformed (unclosed tags, invalid entities, HTML‑style self‑closing tags). Fix: use a strict XML parser with error handling and optional “soft” recovery, or validate the feed before parsing. 7. IP/rate‑based blocking. Some sites limit request frequency from a single IP. Fix: add random delay between requests, limit concurrency, keep a simple request counter, and log HTTP codes 429/403. 8. SSL and certificates. SSL issues (self‑signed certificates, outdated ciphers, hostname mismatch) lead to connection failures. Fix: verify that SSL verification is not disabled in production; ensure test environments reflect real conditions. 9. Feed format. Besides RSS (2.0, 0.9x) and Atom, mixed or non‑standard variants exist. Fix: support multiple feed types, detect the type from the XML structure, and use appropriate parsers. 10. Dynamic feeds and JS rendering. Some “feeds” are HTML pages that become RSS in the browser via JavaScript, or they are not real RSS but HTML with a feed link in meta. Fix: check Content‑Type and document structure; do not attempt to parse HTML as RSS. ####Recommended improvements * Add logging: URL, method, headers (excluding secrets), response code, request duration, exception type. * Implement retry logic with exponential backoff for transient errors (5xx, timeouts, Connection reset). * Limit concurrent requests per domain and add basic rate‑limiting. * Unify error handling: separate error types for “network”, “HTTP”, “parsing”, “invalid format”. * Test against a set of “problematic” feeds (including those that open in a browser but fail in the aggregator).

serrebi commented

2026-06-28 16:45:52 -07:00

Author

Owner

@tseykovets Thanks for the report. I will address this tomorrow. Thanks for waiting and your understanding. I intend to fix this.

@tseykovets Thanks for the report. I will address this tomorrow. Thanks for waiting and your understanding. I intend to fix this.

serrebi commented

2026-06-29 11:16:29 -07:00

Author

Owner

Fixed and released in v1.70.3:

https://github.com/serrebidev/BlindRSS/releases/tag/v1.70.3

What changed:

Local refresh now supports JSON Feed 1.0/1.1 in addition to the existing RSS 0.90/0.91/0.92/0.93/0.94, RSS 1.0, RSS 2.0, and Atom 1.0 parsing path.
Feeds with stale ETag/Last-Modified metadata but no stored articles no longer get stuck at a 304 response with an empty article list; BlindRSS skips conditional headers when the local article cache is empty.
200 OK HTML/challenge/error pages that parse to zero feed entries are now treated as feed errors instead of successful empty refreshes, so bad WAF/proxy responses do not silently replace feed state.
Added regression coverage for JSON Feed, realistic APKMirror and Grav RSS shapes, stale-validator empty-cache recovery, and non-feed HTML responses.

Validation performed before release:

Full test suite: 873 passed, 7 skipped.
Live local-provider check:
- APKMirror Android Accessibility Suite feed: 10 articles inserted
- Grav blog RSS feed: 10 articles inserted
- GNOME Orca GitLab Atom tags feed: 20 articles inserted

Windows release assets and updater manifest are published on the release. The macOS/Linux release workflow was dispatched by the release script and should attach those platform assets after it completes.

Fixed and released in v1.70.3: https://github.com/serrebidev/BlindRSS/releases/tag/v1.70.3 What changed: - Local refresh now supports JSON Feed 1.0/1.1 in addition to the existing RSS 0.90/0.91/0.92/0.93/0.94, RSS 1.0, RSS 2.0, and Atom 1.0 parsing path. - Feeds with stale ETag/Last-Modified metadata but no stored articles no longer get stuck at a 304 response with an empty article list; BlindRSS skips conditional headers when the local article cache is empty. - 200 OK HTML/challenge/error pages that parse to zero feed entries are now treated as feed errors instead of successful empty refreshes, so bad WAF/proxy responses do not silently replace feed state. - Added regression coverage for JSON Feed, realistic APKMirror and Grav RSS shapes, stale-validator empty-cache recovery, and non-feed HTML responses. Validation performed before release: - Full test suite: 873 passed, 7 skipped. - Live local-provider check: - APKMirror Android Accessibility Suite feed: 10 articles inserted - Grav blog RSS feed: 10 articles inserted - GNOME Orca GitLab Atom tags feed: 20 articles inserted Windows release assets and updater manifest are published on the release. The macOS/Linux release workflow was dispatched by the release script and should attach those platform assets after it completes.

serrebi commented

2026-06-29 14:26:03 -07:00

Author

Owner

Unfortunately, I’m still unable to fetch some feeds in BlindRSS. The problematic URLs are:

Below is the log from BlindRSS when trying to add/read the APKMirror feed. You can see the connection is being forcibly reset by the remote host (ConnectionResetError(10054)), and after a retry, the feed processing fails.

2026-06-29 23:34:47,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): www.apkmirror.com:443
2026-06-29 23:34:48,745 - providers.local - INFO - Local feed refresh start id=399e457d-f25f-4c6e-87ad-63897d6e57f9 title='https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/' force=True respect_cooldown=False conditional=False has_etag=False has_last_modified=False timeout_s=15 retries=1 url=https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/
2026-06-29 23:34:48,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): www.apkmirror.com:443
2026-06-29 23:34:48,797 - providers.local - INFO - Local feed refresh retrying id=399e457d-f25f-4c6e-87ad-63897d6e57f9 title='https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/' attempt=1/2 backoff_s=1.00 error="HTTP Error: ('Connection aborted.', ConnectionResetError(10054, 'The remote host forcibly closed an existing connection', None, 10054, None))" url=https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/
2026-06-29 23:34:49,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): www.apkmirror.com:443
2026-06-29 23:34:49,851 - providers.local - ERROR - Error processing feed https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/: ('Connection aborted.', ConnectionResetError(10054, 'The remote host forcibly closed an existing connection', None, 10054, None))
2026-06-29 23:34:49,860 - providers.local - INFO - Local feed refresh finished id=399e457d-f25f-4c6e-87ad-63897d6e57f9 title='https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/' status=error force=True conditional=False entries=None new_items=0 unread=0 duration_s=1.12 error="HTTP Error: ('Connection aborted.', ConnectionResetError(10054, 'The remote host forcibly closed an existing connection', None, 10054, None))" url=https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/

Important observation:

The URL opens fine in a browser.
If I download the XML via a browser and feed it to BlindRSS from another server, BlindRSS parses it successfully.

This strongly suggests the issue isn’t with XML parsing, but with the HTTP request made by BlindRSS: the server is dropping the connection, likely because the request looks “bot‑like” or non‑browser‑like.

Suggested improvements to address this

To make BlindRSS’s requests look more like those from a real browser and reduce the chance of being blocked, I’d suggest the following:

Set a realistic User-Agent header. Use a modern browser UA string (e.g., Chrome on Windows/macOS) instead of the default/empty one. Many servers block or reset connections based on suspicious or missing UA.
Include additional standard headers. Adding Accept, Accept-Language, and Referer can help the request appear more legitimate.
Respect rate limits and add smarter backoff. Increase the retry delay after failures (e.g., exponential backoff) and avoid hammering the server with rapid successive requests.
Ensure up‑to‑date TLS support. Make sure BlindRSS uses TLS 1.2 or 1.3 and doesn’t rely on outdated protocols; some servers immediately drop connections from clients with old TLS stacks.
Add optional per‑feed configuration. Allow users to specify custom headers or timeouts for specific feeds, in case certain sites need special handling.
Log the full outgoing request (headers, not just URL). This would help debug which part of the request triggers the reset.

Let me know if you’d like any further details.

Thanks for your attention to this issue.

Unfortunately, I’m still unable to fetch some feeds in BlindRSS. The problematic URLs are: * <https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/> * <https://getgrav.org/blog.rss> Below is the log from BlindRSS when trying to add/read the APKMirror feed. You can see the connection is being forcibly reset by the remote host (ConnectionResetError(10054)), and after a retry, the feed processing fails. ``` 2026-06-29 23:34:47,810 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): www.apkmirror.com:443 2026-06-29 23:34:48,745 - providers.local - INFO - Local feed refresh start id=399e457d-f25f-4c6e-87ad-63897d6e57f9 title='https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/' force=True respect_cooldown=False conditional=False has_etag=False has_last_modified=False timeout_s=15 retries=1 url=https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/ 2026-06-29 23:34:48,748 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): www.apkmirror.com:443 2026-06-29 23:34:48,797 - providers.local - INFO - Local feed refresh retrying id=399e457d-f25f-4c6e-87ad-63897d6e57f9 title='https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/' attempt=1/2 backoff_s=1.00 error="HTTP Error: ('Connection aborted.', ConnectionResetError(10054, 'The remote host forcibly closed an existing connection', None, 10054, None))" url=https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/ 2026-06-29 23:34:49,803 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): www.apkmirror.com:443 2026-06-29 23:34:49,851 - providers.local - ERROR - Error processing feed https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/: ('Connection aborted.', ConnectionResetError(10054, 'The remote host forcibly closed an existing connection', None, 10054, None)) 2026-06-29 23:34:49,860 - providers.local - INFO - Local feed refresh finished id=399e457d-f25f-4c6e-87ad-63897d6e57f9 title='https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/' status=error force=True conditional=False entries=None new_items=0 unread=0 duration_s=1.12 error="HTTP Error: ('Connection aborted.', ConnectionResetError(10054, 'The remote host forcibly closed an existing connection', None, 10054, None))" url=https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/ ``` Important observation: * The URL opens fine in a browser. * If I download the XML via a browser and feed it to BlindRSS from another server, BlindRSS parses it successfully. This strongly suggests the issue isn’t with XML parsing, but with the HTTP request made by BlindRSS: the server is dropping the connection, likely because the request looks “bot‑like” or non‑browser‑like. #### Suggested improvements to address this To make BlindRSS’s requests look more like those from a real browser and reduce the chance of being blocked, I’d suggest the following: * Set a realistic User-Agent header. Use a modern browser UA string (e.g., Chrome on Windows/macOS) instead of the default/empty one. Many servers block or reset connections based on suspicious or missing UA. * Include additional standard headers. Adding Accept, Accept-Language, and Referer can help the request appear more legitimate. * Respect rate limits and add smarter backoff. Increase the retry delay after failures (e.g., exponential backoff) and avoid hammering the server with rapid successive requests. * Ensure up‑to‑date TLS support. Make sure BlindRSS uses TLS 1.2 or 1.3 and doesn’t rely on outdated protocols; some servers immediately drop connections from clients with old TLS stacks. * Add optional per‑feed configuration. Allow users to specify custom headers or timeouts for specific feeds, in case certain sites need special handling. * Log the full outgoing request (headers, not just URL). This would help debug which part of the request triggers the reset. Let me know if you’d like any further details. Thanks for your attention to this issue.

serrebi commented

2026-06-29 20:36:17 -07:00

Author

Owner

@tseykovets Thanks again for the excellent diagnosis — the ConnectionResetError(10054) together with the feed loading fine in a browser was the key clue. You were right that this is the HTTP request looking non‑browser‑like at the connection level, not a parsing problem. This is now addressed in v1.70.4 and v1.70.5:

https://github.com/serrebidev/BlindRSS/releases/tag/v1.70.5

What changed:

BlindRSS now sends a complete modern‑Chrome request fingerprint — a realistic User‑Agent plus Accept-Language, Accept-Encoding, the Sec-Fetch-* / sec-ch-ua client hints, Upgrade-Insecure-Requests, Connection, and a site‑root Referer.
Added a real browser TLS/JA3 + HTTP/2 impersonation fallback (via curl_cffi). Many anti‑bot WAFs — APKMirror included — reset the connection based on the TLS handshake fingerprint, before any HTTP headers are read, which is exactly the ConnectionResetError(10054) you saw. Plain Python requests can't get past that no matter which headers it sends. When a fetch is reset, or is met with a Cloudflare/JS challenge or an HTML interstitial, BlindRSS now retries replaying a genuine Chrome fingerprint.
Retry backoff is now exponential (1s → 2s → 4s → 8s) and honors Retry-After, so it backs off politely instead of hammering.
Per‑feed configuration (Feed Properties): for a specific feed you can now set custom request headers, a request timeout, the browser‑impersonation mode (Auto / Always / Never), and an HTTP/HTTPS/SOCKS proxy.
Outgoing requests are logged at DEBUG (method, URL, transport, and headers with secrets redacted) to make this kind of issue diagnosable.

One honest note about your setup: you mentioned running BlindRSS from a server. Some sites (APKMirror is a likely example) reject datacenter/cloud IP ranges by reputation regardless of how browser‑like the request is — at that point no client‑side change, headers or TLS impersonation, can help, because the block is on the IP rather than the request. For that case, set a Proxy on the feed in Feed Properties (a residential/mobile proxy) so the fetch goes out through an accepted IP. You can also set Browser impersonation: Always on a stubborn feed to skip straight to the Chrome fingerprint on the first try.

If APKMirror or Grav still won't load after updating, please enable debug logging, try the feed once, and share the log line for that fetch (it now includes the outgoing request) — and let me know whether the machine is on a datacenter IP. That will tell us immediately whether it's a fingerprint block (fixable in‑app) or an IP‑reputation block (needs the proxy option).

Thanks for your patience and the thorough reports.

@tseykovets Thanks again for the excellent diagnosis — the `ConnectionResetError(10054)` together with the feed loading fine in a browser was the key clue. You were right that this is the HTTP request looking non‑browser‑like at the connection level, not a parsing problem. This is now addressed in v1.70.4 and v1.70.5: https://github.com/serrebidev/BlindRSS/releases/tag/v1.70.5 What changed: - BlindRSS now sends a complete modern‑Chrome request fingerprint — a realistic User‑Agent plus `Accept-Language`, `Accept-Encoding`, the `Sec-Fetch-*` / `sec-ch-ua` client hints, `Upgrade-Insecure-Requests`, `Connection`, and a site‑root `Referer`. - Added a real browser TLS/JA3 + HTTP/2 impersonation fallback (via `curl_cffi`). Many anti‑bot WAFs — APKMirror included — reset the connection based on the TLS handshake fingerprint, *before any HTTP headers are read*, which is exactly the `ConnectionResetError(10054)` you saw. Plain Python `requests` can't get past that no matter which headers it sends. When a fetch is reset, or is met with a Cloudflare/JS challenge or an HTML interstitial, BlindRSS now retries replaying a genuine Chrome fingerprint. - Retry backoff is now exponential (1s → 2s → 4s → 8s) and honors `Retry-After`, so it backs off politely instead of hammering. - Per‑feed configuration (Feed Properties): for a specific feed you can now set custom request headers, a request timeout, the browser‑impersonation mode (Auto / Always / Never), and an HTTP/HTTPS/SOCKS proxy. - Outgoing requests are logged at DEBUG (method, URL, transport, and headers with secrets redacted) to make this kind of issue diagnosable. One honest note about your setup: you mentioned running BlindRSS from a server. Some sites (APKMirror is a likely example) reject datacenter/cloud IP ranges by reputation regardless of how browser‑like the request is — at that point no client‑side change, headers or TLS impersonation, can help, because the block is on the IP rather than the request. For that case, set a **Proxy** on the feed in Feed Properties (a residential/mobile proxy) so the fetch goes out through an accepted IP. You can also set **Browser impersonation: Always** on a stubborn feed to skip straight to the Chrome fingerprint on the first try. If APKMirror or Grav still won't load after updating, please enable debug logging, try the feed once, and share the log line for that fetch (it now includes the outgoing request) — and let me know whether the machine is on a datacenter IP. That will tell us immediately whether it's a fingerprint block (fixable in‑app) or an IP‑reputation block (needs the proxy option). Thanks for your patience and the thorough reports.

serrebi closed this issue

2026-06-30 06:06:14 -07:00

serrebi commented

2026-06-30 06:06:14 -07:00

Author

Owner

I tested it on version 1.70.6.

The problem with the feed https://getgrav.org/blog.rss was resolved immediately. This feed loads and can be read immediately after being added.

The feed https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/ couldn't be read immediately after being added.
However, if I change the "Browser impersonation:" setting in its properties from "Auto" to "Always," the feed can be read.

A similar situation exists with feeds provided by feedburner.com.

Overall, I consider the problem resolved and am closing this issue.

Great job! Thank you very much!

For reference, here is the log of an unsuccessful feed loading from apkmirror when the "Browser impersonation:" setting is set to "Auto":

2026-06-30 13:57:34,131 - core.utils - DEBUG - HTTP GET https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/ via requests headers={'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36', 'Accept': 'application/rss+xml,application/xml,application/atom+xml,text/xml;q=0.9,*/*;q=0.8', 'Accept-Language': 'en-US,en;q=0.9', 'Accept-Encoding': 'gzip, deflate, br', 'Upgrade-Insecure-Requests': '1', 'sec-ch-ua': '"Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': '"Windows"', 'Sec-Fetch-Dest': 'document', 'Sec-Fetch-Mode': 'navigate', 'Sec-Fetch-Site': 'none', 'Sec-Fetch-User': '?1', 'Connection': 'keep-alive'}
2026-06-30 13:57:34,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): www.apkmirror.com:443
2026-06-30 13:57:38,219 - core.utils - DEBUG - HTTP GET https://api.github.com/repos/serrebidev/BlindRSS/releases/latest via requests headers={'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36', 'Accept': 'application/vnd.github+json', 'Accept-Language': 'en-US,en;q=0.9', 'Accept-Encoding': 'gzip, deflate, br', 'Upgrade-Insecure-Requests': '1', 'sec-ch-ua': '"Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': '"Windows"', 'Sec-Fetch-Dest': 'document', 'Sec-Fetch-Mode': 'navigate', 'Sec-Fetch-Site': 'none', 'Sec-Fetch-User': '?1', 'Connection': 'keep-alive'}
2026-06-30 13:57:38,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.github.com:443
2026-06-30 13:57:38,567 - urllib3.connectionpool - DEBUG - https://api.github.com:443 "GET /repos/serrebidev/BlindRSS/releases/latest HTTP/1.1" 200 2118
2026-06-30 13:57:44,220 - providers.local - INFO - Local feed refresh start id=065bcf85-563e-4b83-9109-3466486a62ac title='https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/' force=True respect_cooldown=False conditional=False has_etag=False has_last_modified=False timeout_s=15 retries=1 url=https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/
2026-06-30 13:57:44,224 - core.utils - DEBUG - HTTP GET https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/ via requests headers={'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36', 'Accept': 'application/rss+xml,application/xml,application/atom+xml,text/xml;q=0.9,*/*;q=0.8', 'Accept-Language': 'en-US,en;q=0.9', 'Accept-Encoding': 'gzip, deflate, br', 'Upgrade-Insecure-Requests': '1', 'sec-ch-ua': '"Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': '"Windows"', 'Sec-Fetch-Dest': 'document', 'Sec-Fetch-Mode': 'navigate', 'Sec-Fetch-Site': 'none', 'Sec-Fetch-User': '?1', 'Connection': 'keep-alive', 'Cache-Control': 'no-cache, max-age=0', 'Pragma': 'no-cache', 'Expires': '0', 'Referer': 'https://www.apkmirror.com/'}
2026-06-30 13:57:44,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): www.apkmirror.com:443
2026-06-30 13:57:59,257 - providers.local - INFO - Local feed refresh retrying id=065bcf85-563e-4b83-9109-3466486a62ac title='https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/' attempt=1/3 backoff_s=1.00 error="HTTP Error: HTTPSConnectionPool(host='www.apkmirror.com', port=443): Read timed out. (read timeout=15)" url=https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/
2026-06-30 13:58:00,263 - core.utils - DEBUG - HTTP GET https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/ via requests headers={'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36', 'Accept': 'application/rss+xml,application/xml,application/atom+xml,text/xml;q=0.9,*/*;q=0.8', 'Accept-Language': 'en-US,en;q=0.9', 'Accept-Encoding': 'gzip, deflate, br', 'Upgrade-Insecure-Requests': '1', 'sec-ch-ua': '"Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': '"Windows"', 'Sec-Fetch-Dest': 'document', 'Sec-Fetch-Mode': 'navigate', 'Sec-Fetch-Site': 'none', 'Sec-Fetch-User': '?1', 'Connection': 'keep-alive', 'Cache-Control': 'no-cache, max-age=0', 'Pragma': 'no-cache', 'Expires': '0', 'Referer': 'https://www.apkmirror.com/'}
2026-06-30 13:58:00,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): www.apkmirror.com:443
2026-06-30 13:58:15,309 - providers.local - ERROR - Error processing feed https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/: HTTPSConnectionPool(host='www.apkmirror.com', port=443): Read timed out. (read timeout=15)
2026-06-30 13:58:15,314 - providers.local - INFO - Local feed refresh finished id=065bcf85-563e-4b83-9109-3466486a62ac title='https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/' status=error force=True conditional=False entries=None new_items=0 unread=0 duration_s=31.10 error="HTTP Error: HTTPSConnectionPool(host='www.apkmirror.com', port=443): Read timed out. (read timeout=15)" url=https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/

I tested it on version 1.70.6. The problem with the feed <https://getgrav.org/blog.rss> was resolved immediately. This feed loads and can be read immediately after being added. The feed <https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/> couldn't be read immediately after being added. However, if I change the "Browser impersonation:" setting in its properties from "Auto" to "Always," the feed can be read. A similar situation exists with feeds provided by feedburner.com. Overall, I consider the problem resolved and am closing this issue. Great job! Thank you very much! For reference, here is the log of an unsuccessful feed loading from apkmirror when the "Browser impersonation:" setting is set to "Auto": ``` 2026-06-30 13:57:34,131 - core.utils - DEBUG - HTTP GET https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/ via requests headers={'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36', 'Accept': 'application/rss+xml,application/xml,application/atom+xml,text/xml;q=0.9,*/*;q=0.8', 'Accept-Language': 'en-US,en;q=0.9', 'Accept-Encoding': 'gzip, deflate, br', 'Upgrade-Insecure-Requests': '1', 'sec-ch-ua': '"Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': '"Windows"', 'Sec-Fetch-Dest': 'document', 'Sec-Fetch-Mode': 'navigate', 'Sec-Fetch-Site': 'none', 'Sec-Fetch-User': '?1', 'Connection': 'keep-alive'} 2026-06-30 13:57:34,135 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): www.apkmirror.com:443 2026-06-30 13:57:38,219 - core.utils - DEBUG - HTTP GET https://api.github.com/repos/serrebidev/BlindRSS/releases/latest via requests headers={'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36', 'Accept': 'application/vnd.github+json', 'Accept-Language': 'en-US,en;q=0.9', 'Accept-Encoding': 'gzip, deflate, br', 'Upgrade-Insecure-Requests': '1', 'sec-ch-ua': '"Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': '"Windows"', 'Sec-Fetch-Dest': 'document', 'Sec-Fetch-Mode': 'navigate', 'Sec-Fetch-Site': 'none', 'Sec-Fetch-User': '?1', 'Connection': 'keep-alive'} 2026-06-30 13:57:38,223 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): api.github.com:443 2026-06-30 13:57:38,567 - urllib3.connectionpool - DEBUG - https://api.github.com:443 "GET /repos/serrebidev/BlindRSS/releases/latest HTTP/1.1" 200 2118 2026-06-30 13:57:44,220 - providers.local - INFO - Local feed refresh start id=065bcf85-563e-4b83-9109-3466486a62ac title='https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/' force=True respect_cooldown=False conditional=False has_etag=False has_last_modified=False timeout_s=15 retries=1 url=https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/ 2026-06-30 13:57:44,224 - core.utils - DEBUG - HTTP GET https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/ via requests headers={'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36', 'Accept': 'application/rss+xml,application/xml,application/atom+xml,text/xml;q=0.9,*/*;q=0.8', 'Accept-Language': 'en-US,en;q=0.9', 'Accept-Encoding': 'gzip, deflate, br', 'Upgrade-Insecure-Requests': '1', 'sec-ch-ua': '"Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': '"Windows"', 'Sec-Fetch-Dest': 'document', 'Sec-Fetch-Mode': 'navigate', 'Sec-Fetch-Site': 'none', 'Sec-Fetch-User': '?1', 'Connection': 'keep-alive', 'Cache-Control': 'no-cache, max-age=0', 'Pragma': 'no-cache', 'Expires': '0', 'Referer': 'https://www.apkmirror.com/'} 2026-06-30 13:57:44,232 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): www.apkmirror.com:443 2026-06-30 13:57:59,257 - providers.local - INFO - Local feed refresh retrying id=065bcf85-563e-4b83-9109-3466486a62ac title='https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/' attempt=1/3 backoff_s=1.00 error="HTTP Error: HTTPSConnectionPool(host='www.apkmirror.com', port=443): Read timed out. (read timeout=15)" url=https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/ 2026-06-30 13:58:00,263 - core.utils - DEBUG - HTTP GET https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/ via requests headers={'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/124.0.0.0 Safari/537.36', 'Accept': 'application/rss+xml,application/xml,application/atom+xml,text/xml;q=0.9,*/*;q=0.8', 'Accept-Language': 'en-US,en;q=0.9', 'Accept-Encoding': 'gzip, deflate, br', 'Upgrade-Insecure-Requests': '1', 'sec-ch-ua': '"Chromium";v="124", "Google Chrome";v="124", "Not-A.Brand";v="99"', 'sec-ch-ua-mobile': '?0', 'sec-ch-ua-platform': '"Windows"', 'Sec-Fetch-Dest': 'document', 'Sec-Fetch-Mode': 'navigate', 'Sec-Fetch-Site': 'none', 'Sec-Fetch-User': '?1', 'Connection': 'keep-alive', 'Cache-Control': 'no-cache, max-age=0', 'Pragma': 'no-cache', 'Expires': '0', 'Referer': 'https://www.apkmirror.com/'} 2026-06-30 13:58:00,274 - urllib3.connectionpool - DEBUG - Starting new HTTPS connection (1): www.apkmirror.com:443 2026-06-30 13:58:15,309 - providers.local - ERROR - Error processing feed https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/: HTTPSConnectionPool(host='www.apkmirror.com', port=443): Read timed out. (read timeout=15) 2026-06-30 13:58:15,314 - providers.local - INFO - Local feed refresh finished id=065bcf85-563e-4b83-9109-3466486a62ac title='https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/' status=error force=True conditional=False entries=None new_items=0 unread=0 duration_s=31.10 error="HTTP Error: HTTPSConnectionPool(host='www.apkmirror.com', port=443): Read timed out. (read timeout=15)" url=https://www.apkmirror.com/apk/google-inc/android-accessibility-suite/feed/ ```